Exponential Family Of Distributions Research Articles

Alpha Power Odd Generalized Exponential Family of Distributions: Model, Properties and Applications

In this paper, we exhibit a general family of distributions called alpha power odd generalized exponential family of distributions. The new family is very flexible with increasing, decreasing, J, reversed-J, bathtub shapes. Statistical properties of the family such as quantile, expansion of density function, moments, incomplete moments, mean deviation, Bonferroni and Lorenz curves are proposed. The method of maximum likelihood to estimate the model parameters is used. Three applications based on real data sets are the importance and flexibility of the three proposed models.

Estimation of P(X ≤ Y) for discrete distributions with non-identical support

Abstract The Uniformly Minimum Variance Unbiased (UMVU) and the Maximum Likelihood (ML) estimations of R = P(X ≤ Y) and the associated variance are considered for independent discrete random variables X and Y. Assuming a discrete uniform distribution for X and the distribution of Y as a member of the discrete one parameter exponential family of distributions, theoretical expressions of such quantities are derived. Similar expressions are obtained when X and Y interchange their roles and both variables are from the discrete uniform distribution. A simulation study is carried out to compare the estimators numerically. A real application based on demand-supply system data is provided.

Sensitivity Analysis for Binary Sampling Systems via Quantitative Fisher Information Lower Bounds

Determining the quality of sensing devices exhibiting minimal digitization complexity is addressed. Measurements of such sensor systems are characterized by multivariate binary distributions and assessing sensitivity via the Cramér-Rao lower bound turns out to be intractable. In this context, the Fisher matrix of the exponential family and a lower bound for arbitrary probabilistic models are discussed. The conservative approximation for Fisher’s information matrix rests on a surrogate exponential family distribution connected to the actual data-generating system by two compact equivalences. Without characterizing the likelihood and its support, this probabilistic notion enables designing estimators that consistently achieve the sensitivity level defined by the inverse of the conservative information matrix. For hard-limited multivariate Gaussian signal models, a quadratic exponential surrogate distribution tames statistical complexity such that a quantitative and conservative assessment of Fisher information becomes possible. This result is exploited for the Fisherian quantization loss analysis of an array with low-complexity binary sensors in comparison to an ideal system featuring infinite amplitude resolution. Additionally, data-driven assessment by estimating a conservative approximation for the Fisher matrix under recursive binary sampling as implemented in ΣΔ-modulating analog-to-digital converters is demonstrated.

Two-step Bayesian methods for generalized regression driven by partial differential equations

In certain non-linear regression models, the functional form of the regression function is not explicitly available, but is only described by a set of differential equations. For regression models described by a set of ordinary differential equations (ODEs), both Bayesian and non-Bayesian methods for inference were developed in the literature. In this paper, we consider a Bayesian approach to non-linear regression with respect to a multidimensional predictor variable given by a set of partial differential equations (PDEs). We consider a computationally convenient two-step approach by first representing the functions nonparametrically, constructing a finite random series prior using tensor products of B-splines and directly inducing a posterior distribution on parameter space through an appropriate projection map. By considering three different choices of the projection map, we propose three different approaches with their merits. We allow generalized non-linear regression with the response variable following an exponential family of distributions, extending the method beyond regression with additive normal errors. We establish Bernstein-von Mises type theorems which show n-consistency and asymptotically correct frequentist coverage of Bayesian credible regions. We also conduct a simulation study to evaluate the finite sample performances of the proposed methods.

Modeling and inference for multivariate time series of counts based on the INGARCH scheme

Modeling multivariate time series of counts using the integer-valued generalized autoregressive conditional heteroscedastic (INGARCH) scheme is proposed. The key idea is to model each component of the time series with a univariate INGARCH model, where the conditional distribution is modeled with a one-parameter exponential family distribution, and to use a (nonlinear) parametric function of all components to recursively produce the conditional means. It is shown that the proposed multivariate INGARCH (MINGARCH) model is strictly stationary and ergodic. For inference, the quasi-maximum likelihood estimator (QMLE) and the minimum density power divergence estimator (MDPDE) for robust estimation are adopted, and their consistency and asymptotic normality are verified. As an application, the change point test based on the QMLE and MDPDE is illustrated. The Monte Carlo simulation study and real data analysis using the number of weekly syphilis cases in the United States are conducted to confirm the validity of the proposed method.

The Generalized Odd Linear Exponential Family of Distributions with Applications to Reliability Theory

A new family of continuous distributions called the generalized odd linear exponential family is proposed. The probability density and cumulative distribution function are expressed as infinite linear mixtures of exponentiated-F distribution. Important statistical properties such as quantile function, moment generating function, distribution of order statistics, moments, mean deviations, asymptotes and the stress–strength model of the proposed family are investigated. The maximum likelihood estimation of the parameters is presented. Simulation is carried out for two of the mentioned sub-models to check the asymptotic behavior of the maximum likelihood estimates. Two real-life data sets are used to establish the credibility of the proposed model. This is achieved by conducting data fitting of two of its sub-models and then comparing the results with suitable competitive lifetime models to generate conclusive evidence.

Nonparametric discriminant analysis with network structures in predictor

ABSTRACT Multiclassification, known as classification for multi-label responses, has been an important problem in supervised learning and has attracted our attention. Discriminant analysis (DA) is a popular method to deal with multiclassification. With the increasing availability of complex data, it becomes more challenging to analyse them. One of the important features in complex data is the network structure, which is ubiquitous in high-dimensional data because of strong or weak correlations among variables. In addition, in the framework of DA, an assumption of normal distributions is imposed on the predictors, but it is usually invalid in applications. To relax the normality assumption, we propose a nonparametric discriminant function to address multiclassification. In addition, to incorporate the network structure and improve the accuracy of classification, we develop three different network-based surrogate predictors to replace conventional predictors. The key features of the proposed method include the incorporation of network structures in predictors and allowance of predictors to follow exponential family distributions. Finally, numerical studies, including simulation and real data analysis, are conducted to assess the performance of the proposed method.

Calculating and Comparing the Annualized Relapse Rate and Estimating the Confidence Interval in Relapsing Neurological Diseases.

Calculating the crude or adjusted annualized relapse rate (ARR) and its confidence interval (CI) is often required in clinical studies to evaluate chronic relapsing diseases, such as multiple sclerosis and neuromyelitis optica spectrum disorders. However, accurately calculating ARR and estimating the 95% CI requires careful application of statistical approaches and basic familiarity with the exponential family of distributions. When the relapse rate can be regarded as constant over time or by individuals, the crude ARR can be calculated using the person-years method, which divides the number of all observed relapses among all participants by the total follow-up period of the study cohort. If the number of relapses can be modeled by the Poisson distribution, the 95% CI of ARR can be obtained by finding the 2.5% upper and lower critical values of the parameter λ as the mean. Basic familiarity with F-statistics is also required when comparing the ARR between two disease groups. It is necessary to distinguish the observed relapse rate ratio (RR) between two sample groups (sample RR) from the unobserved RR between their originating populations (population RR). The ratio of population RR to sample RR roughly follows the F distribution, with degrees of freedom obtained by doubling the number of observed relapses in the two sample groups. Based on this, a 95% CI of the population RR can be estimated. When the count data of the response variable is overdispersed, the negative binomial distribution would be a better fit than the Poisson. Adjusted ARR and the 95% CI can be obtained by using the generalized linear regression models after selecting appropriate error structures (e.g., Poisson, negative binomial, zero-inflated Poisson, and zero-inflated negative binomial) according to the overdispersion and zero-inflation in the response variable.

Natural Discrete One Parameter Polynomial Exponential Family of Distributions and the Application

Natural Discrete One Parameter Polynomial Exponential Family of Distributions and the Application

A generalized interrupted time series model for assessing complex health care interventions.

Assessing the impact of complex interventions on measurable health outcomes is a growing concern in health care and health policy. Interrupted time series (ITS) designs borrow from traditional case-crossover designs and function as quasi-experimental methodology able to retrospectively analyze the impact of an intervention. Statistical models used to analyze ITS designs primarily focus on continuous-valued outcomes. We propose the "Generalized Robust ITS" (GRITS) model appropriate for outcomes whose underlying distribution belongs to the exponential family of distributions, thereby expanding the available methodology to adequately model binary and count responses. GRITS formally implements a test for the existence of a change point in discrete ITS. The methodology proposed is able to test for the existence of and estimate the change point, borrow information across units in multi-unit settings, and test for differences in the mean function and correlation pre- and post-intervention. The methodology is illustrated by analyzing patient falls from a hospital that implemented and evaluated a new care delivery model in multiple units.

Efficient Optimization of Partition Scan Statistics via the Consecutive Partitions Property

We generalize the spatial and subset scan statistics from the single to the multiple subset case. The two main approaches to defining the log-likelihood ratio statistic in the single subset case – the population-based and expectation-based scan statistics – are considered, leading to risk partitioning and multiple cluster detection scan statistics, respectively. We show that, for distributions in a separable exponential family, the risk partitioning scan statistic can be expressed as a scaled f-divergence of the normalized count and baseline vectors, and the multiple cluster detection scan statistic as a sum of scaled Bregman divergences. In either case, however, maximization of the scan statistic by exhaustive search over all partitionings of the data requires exponential time. To make this optimization computationally feasible, we prove sufficient conditions under which the optimal partitioning is guaranteed to be consecutive. This Consecutive Partitions Property generalizes the linear-time subset scanning property from two partitions (the detected subset and the remaining data elements) to the multiple partition case. While the number of consecutive partitionings of n elements into t partitions scales as , making it computationally expensive for large t, we present a dynamic programming approach which identifies the optimal consecutive partitioning in time, thus allowing for the exact and efficient solution of large-scale risk partitioning and multiple cluster detection problems. Finally, we demonstrate the detection performance and practical utility of partition scan statistics using simulated and real-world data.

On the odd inverse exponential class of distributions: Properties, applications and cure fraction regression

In this paper, the generalized odd inverse exponential family of distributions is developed and studied. Statistical properties such as moments, moment generating function, incomplete moment, characteristic function and order statistics of the new family were derived. Two special distributions; generalized odd inverse exponential Weibull and generalized odd inverse exponential Lomax distributions were proposed. Again, regression models with cure fractions were developed using the special distributions. Four applications of the new family using cancer datasets suggest that the special distributions provide better fit to the datasets than the other fitted models.

Posterior probabilities: Nonmonotonicity, asymptotic rates, log-concavity, and Turán’s inequality

In the standard Bayesian framework data are assumed to be generated by a distribution parametrized by $\theta$ in a parameter space $\Theta$, over which a prior distribution $\pi$ is given. A Bayesian statistician quantifies the belief that the true parameter is $\theta_{0}$ in $\Theta$ by its posterior probability given the observed data. We investigate the behavior of the posterior belief in $\theta_{0}$ when the data are generated under some parameter $\theta_{1},$ which may or may not be the same as $\theta_{0}.$ Starting from stochastic orders, specifically, likelihood ratio dominance, that obtain for resulting distributions of posteriors, we consider monotonicity properties of the posterior probabilities as a function of the sample size when data arrive sequentially. While the $\theta_{0}$-posterior is monotonically increasing (i.e., it is a submartingale) when the data are generated under that same $\theta_{0}$, it need not be monotonically decreasing in general, not even in terms of its overall expectation, when the data are generated under a different $\theta_{1}.$ In fact, it may keep going up and down many times, even in simple cases such as iid coin tosses. We obtain precise asymptotic rates when the data come from the wide class of exponential families of distributions; these rates imply in particular that the expectation of the $\theta_{0}$-posterior under $\theta_{1}\neq\theta_{0}$ is eventually strictly decreasing. Finally, we show that in a number of interesting cases this expectation is a log-concave function of the sample size, and thus unimodal. In the Bernoulli case we obtain this by developing an inequality that is related to Tur\'{a}n's inequality for Legendre polynomials.

Mixtures of Semi-Parametric Generalised Linear Models

The mixture of generalised linear models (MGLM) requires knowledge about each mixture component’s specific exponential family (EF) distribution. This assumption is relaxed and a mixture of semi-parametric generalised linear models (MSPGLM) approach is proposed, which allows for unknown distributions of the EF for each mixture component while much of the parametric structure of the traditional MGLM is retained. Such an approach inherently allows for both symmetric and non-symmetric component distributions, frequently leading to non-symmetrical response variable distributions. It is assumed that the random component of each mixture component follows an unknown distribution of the EF. The specific member can either be from the standard class of distributions or from the broader set of admissible distributions of the EF which is accessible through the semi-parametric procedure. Since the inverse link functions of the mixture components are unknown, the MSPGLM estimates each mixture component’s inverse link function using a kernel smoother. The MSPGLM algorithm alternates the estimation of the regression parameters with the estimation of the inverse link functions. The properties of the proposed MSPGLM are illustrated through a simulation study on the separable individual components. The MSPGLM procedure is also applied on two data sets.

Interpretability of Neural Networks with Probability Density Functions

AbstractIt is an interesting topic to interpret artificial neural networks (ANNs) by considering some change various approaches. This paper explores the relationship between the input and output units of the simplest ANN, a single layer perceptron for the binary classification problem, from the probability point of view. If the feature variables of datasets follow independent normal distribution and outputs are activated by sigmoid function or smooth Relu function, we advocate that the probability density function (pdf) of the output variable is an exponential family distribution. Furthermore, by introducing an intermediate variable, the pdf of the output variable can be written as a linear combination of three normal distributions with same spread but different centers. Based on these results, the probability of the predicted class label can be written as a standard normal cumulative distribution function (cdf). The originality of this paper comes with interesting theoretical results to provide ANNs with a new description of the relationship between input variables to output variables, which can enable ANNs to be understood from a new perspective. Extensive experiments based on one artificial synthesized dataset and ten real‐world benchmark datasets validate the reasonability of those results.

Efficient integration of aggregate data and individual participant data in one-way mixed models.

Often both aggregate data (AD) studies and individual participant data (IPD) studies are available for specific treatments. Combining these two sources of data could improve the overall meta-analytic estimates of treatment effects. Moreover, often for some studies with AD, the associated IPD maybe available, albeit at some extra effort or cost to the analyst. We propose a method for combining treatment effects across trials when the response is from the exponential family of distribution and hence a generalized linear model structure can be used. We consider the case when treatment effects are fixed and common across studies. Using the proposed combination method, we study the relative efficiency of analyzing all IPD studies vs combining various percentages of AD and IPD studies. For many different models, design constraints under which the AD estimators are the IPD estimators, and hence fully efficient, are known. For such models, we advocate a selection procedure that chooses AD studies over IPD studies in a manner that force least departure from design constraints and hence ensures an efficient combined AD and IPD estimator.

Determination of Hazard State of Non-Communicable Diseases Using Semi-Markov Model

Background: The developed Semi-Markov model with Kumaraswamy Exponentiated Inverse Rayleigh distribution examined patients with hypertension, heart diseases, smoking habits and Stroke, is measured from one state to another. Materials and Methods: Patients with Non-Communicable disease described through Kumaraswamy Exponentiated Inverse Rayleigh distribution. Results: The estimated parameters of Semi-Markov model with this distribution predicted by the maximum likelihood estimation for each successive state observed significant abnormality. The data noted predicts established model is a good fit for many attributes that prevailed in studied data. The developed Semi-Markov model is a best fit for non-Communicable disease in the long run of patient’s data. Through different Exponential family distribution, one can look at for further perfect fit of patient data, which is to be estimated. Conclusion: This model can be an alternative method to estimate the effect of patient in survival analysis, where it will be effective in time consumption in medical field. Keywords: heart diseases, hypertension, Semi-Markov processes, smoking, stroke.

Statistical Inference for a General Family of Modified Exponentiated Distributions

In this paper, a modified exponentiated family of distributions is introduced. The new model was built from a continuous parent cumulative distribution function and depends on a shape parameter. Its most relevant characteristics have been obtained: the probability density function, quantile function, moments, stochastic ordering, Poisson mixture with our proposal as the mixing distribution, order statistics, tail behavior and estimates of parameters. We highlight the particular model based on the classical exponential distribution, which is an alternative to the exponentiated exponential, gamma and Weibull. A simulation study and a real application are presented. It is shown that the proposed family of distributions is of interest to applied areas, such as economics, reliability and finances.

A note on the coverage behaviour of bootstrap percentile confidence intervals for constrained parameters

The asymptotic behaviour of the commonly used bootstrap percentile confidence interval is investigated when the parameters are subject to linear inequality constraints. We concentrate on the important one- and two-sample problems with data generated from general parametric distributions in the natural exponential family. The focus of this paper is on quantifying the coverage probabilities of the parametric bootstrap percentile confidence intervals, in particular their limiting behaviour near boundaries. We propose a local asymptotic framework to study this subtle coverage behaviour. Under this framework, we discover that when the true parameters are on, or close to, the restriction boundary, the asymptotic coverage probabilities can always exceed the nominal level in the one-sample case; however, they can be, remarkably, both under and over the nominal level in the two-sample case. Using illustrative examples, we show that the results provide theoretical justification and guidance on applying the bootstrap percentile method to constrained inference problems.

Mendelian randomization in the multivariate general linear model framework.

Mendelian randomization (MR) is an application of instrumental variable (IV) methods to observational data in which the IV is a genetic variant. MR methods applicable to the general exponential family of distributions are currently not well characterized. We adapt a general linear model framework to the IV setting and propose a general MR method applicable to any full-rank distribution from the exponential family. Empirical bias and coverage are estimated via simulations. The proposed method is compared to several existing MR methods. Real data analyses are performed using data from the REGARDS study to estimate the potential causal effect of smoking frequency on stroke risk in African Americans. In simulations with binary variates and very weak instruments the proposed method had the lowest median [Q1 , Q3 ] bias (0.10 [-3.68 to 3.62]); compared with 2SPS (0.27 [-3.74 to 4.26]) and the Wald method (-0.69 [-1.72 to 0.35]). Low bias was observed throughout other simulation scenarios; as well as more than 90% coverage for the proposed method. In simulations with count variates, the proposed method performed comparably to 2SPS; the Wald method maintained the most consistent low bias; and 2SRI was biased towards the null. Real data analyses find no evidence for a causal effect of smoking frequency on stroke risk. The proposed MR method has low bias and acceptable coverage across a wide range of distributional scenarios and instrument strengths; and provides a more parsimonious framework for asymptotic hypothesis testing compared to existing two-stage procedures.