Semiparametric Density Estimation Research Articles

A refreshing take on the inverted Dirichlet via a mode parameterization with some statistical illustrations

AbstractThe inverted Dirichlet (IDir) distribution is a popular choice for modeling multivariate data with positive support; however, its conventional parameterization can be challenging to interpret. In this paper, we propose a refreshing take on the IDir distribution through a convenient mode-based parameterization, resulting in the mode-reparameterized IDir (mIDir). This new parameterization aims to enhance the use of the IDir in various contexts. We provide relevant statistical illustrations in robust and nonparametric statistics, model-based clustering, and semiparametric density estimation, all benefiting from this novel perspective on the IDir for computation and implementation. First, we define finite mIDir mixtures for clustering and semiparametric density estimation. Secondly, we introduce a smoother based on mIDir kernels, which, by design, avoids allocating probability mass to unrealistic negative values, thereby addressing the boundary bias issue. Thirdly, we introduce a heavy-tailed generalization of the mIDir distribution, referred to as the contaminated mIDir (cmIDir), which effectively handles and detects mild outliers, making it suitable for robust statistics. Maximum likelihood estimates of the parameters for the parametric models are obtained using a developed EM algorithm as well as direct numerical optimization. A parameter recovery analysis demonstrates the successful application of the estimation method, while a sensitivity analysis examines the impact of mild outliers on both the mIDir and cmIDir models. The flexibility and advantages of the proposed mIDir-based models are showcased through several real data analyses and illustrations.

Semi-Parametric Estimation Using Bernstein Polynomial and a Finite Gaussian Mixture Model.

The central focus of this paper is upon the alleviation of the boundary problem when the probability density function has a bounded support. Mixtures of beta densities have led to different methods of density estimation for data assumed to have compact support. Among these methods, we mention Bernstein polynomials which leads to an improvement of edge properties for the density function estimator. In this paper, we set forward a shrinkage method using the Bernstein polynomial and a finite Gaussian mixture model to construct a semi-parametric density estimator, which improves the approximation at the edges. Some asymptotic properties of the proposed approach are investigated, such as its probability convergence and its asymptotic normality. In order to evaluate the performance of the proposed estimator, a simulation study and some real data sets were carried out.

Generalized Mixtures of Finite Mixtures and Telescoping Sampling

Within a Bayesian framework, a comprehensive investigation of mixtures of finite mixtures (MFMs), i.e., finite mixtures with a prior on the number of components, is performed. This model class has applications in model-based clustering as well as for semi-parametric density estimation and requires suitable prior specifications and inference methods to exploit its full potential. We contribute by considering a generalized class of MFMs where the hyperparameter γK of a symmetric Dirichlet prior on the weight distribution depends on the number of components. We show that this model class may be regarded as a Bayesian non-parametric mixture outside the class of Gibbs-type priors. We emphasize the distinction between the number of components K of a mixture and the number of clusters K+, i.e., the number of filled components given the data. In the MFM model, K+ is a random variable and its prior depends on the prior on K and on the hyperparameter γK. We employ a flexible prior distribution for the number of components K and derive the corresponding prior on the number of clusters K+ for generalized MFMs. For posterior inference we propose the novel telescoping sampler which allows Bayesian inference for mixtures with arbitrary component distributions without resorting to reversible jump Markov chain Monte Carlo (MCMC) methods. The telescoping sampler explicitly samples the number of components, but otherwise requires only the usual MCMC steps of a finite mixture model. The ease of its application using different component distributions is demonstrated on several data sets.

Gender, Price, and Quantity Effects in U.S. Earnings Inequality: Revisiting Counterfactual Density Estimates

I decompose changes in the U.S. household earnings distribution from 1975 to 2018 to examine the labor market processes underlying its evolution over time. I model the distri- bution of earnings as a function of price effects (wages) and quantity effects (work hours and household employment), each of which are specified separately for men and women, and apply a semi-parametric density estimation technique to infer their contributions to in- equality measures over time. Results indicate that changes to the male wage distribution explain much of the growth in earnings inequality, but that its contribution varied greatly over time, with peak contributions in the mid 1990s; while changes in female work hours have actually mitigated inequality growth, particularly by raising earnings in the lower and mid portions of the distribution, with very consistent effects over time. These results demonstrate the relevance of work hours in addition to wage rates in explaining earnings inequality, and the importance of gender differences therein.

Smoothing Spline Semiparametric Density Models

Density estimation plays a fundamental role in many areas of statistics and machine learning. Parametric, nonparametric, and semiparametric density estimation methods have been proposed in the literature. Semiparametric density models are flexible in incorporating domain knowledge and uncertainty regarding the shape of the density function. Existing literature on semiparametric density models is scattered and lacks a systematic framework. In this article, we consider a unified framework based on reproducing kernel Hilbert space for modeling, estimation, computation, and theory. We propose general semiparametric density models for both a single sample and multiple samples which include many existing semiparametric density models as special cases. We develop penalized likelihood based estimation methods and computational methods under different situations. We establish joint consistency and derive convergence rates of the proposed estimators for both finite dimensional Euclidean parameters and an infinite-dimensional functional parameter. We validate our estimation methods empirically through simulations and an application. Supplementary materials for this article are available online.

A new look at the inverse Gaussian distribution with applications to insurance and economic data

ABSTRACTInsurance and economic data are often positive, and we need to take into account this peculiarity in choosing a statistical model for their distribution. An example is the inverse Gaussian (IG), which is one of the most famous and considered distributions with positive support. With the aim of increasing the use of the IG distribution on insurance and economic data, we propose a convenient mode-based parameterization yielding the reparametrized IG (rIG) distribution; it allows/simplifies the use of the IG distribution in various branches of statistics, and we give some examples. In nonparametric statistics, we define a smoother based on rIG kernels. By construction, the estimator is well-defined and does not allocate probability mass to unrealistic negative values. We adopt likelihood cross-validation to select the smoothing parameter. In robust statistics, we propose the contaminated IG distribution, a heavy-tailed generalization of the rIG distribution to accommodate mild outliers. Finally, for model-based clustering and semiparametric density estimation, we present finite mixtures of rIG distributions. We use the EM algorithm to obtain maximum likelihood estimates of the parameters of the mixture and contaminated models. We use insurance data about bodily injury claims, and economic data about incomes of Italian households, to illustrate the models.

Bayesian estimation of multivariate normal mixtures with covariate-dependent mixing weights, with an application in antimicrobial resistance monitoring.

Bacteria with a reduced susceptibility against antimicrobials pose a major threat to public health. Therefore, large programs have been set up to collect minimum inhibition concentration (MIC) values. These values can be used to monitor the distribution of the nonsusceptible isolates in the general population. Data are collected within several countries and over a number of years. In addition, the sampled bacterial isolates were not tested for susceptibility against one antimicrobial, but rather against an entire range of substances. Interest is therefore in the analysis of the joint distribution of MIC data on two or more antimicrobials, while accounting for a possible effect of covariates. In this regard, we present a Bayesian semiparametric density estimation routine, based on multivariate Gaussian mixtures. The mixing weights are allowed to depend on certain covariates, thereby allowing the user to detect certain changes over, for example, time. The new approach was applied to data collected in Europe in 2010, 2012, and 2013. We investigated the susceptibility of Escherichia coli isolates against ampicillin and trimethoprim, where we found that there seems to be a significant increase in the proportion of nonsusceptible isolates. In addition, a simulation study was carried out, showing the promising behavior of the proposed method in the field of antimicrobial resistance.

Semiparametric smoothing approach to hazard rate estimation

ABSTRACTThis research extends the multiplicative density estimation technique of Naito [(2004), ‘Semiparametric Density Estimation by Local -fitting’, The Annals of Statistics, 32, 1162–1191] to the hazard rate setting. The proposed estimate consists of a parametric estimate of the underlying model times a nonparametric correction factor. The reasoning of this approach is first illustrated by varying the shape parameter involved in the approximation and displaying the benefits of the resulting estimate in an sense for specific example distributions. The sample analogue of this approach is then used as the basis for building an estimator of the true hazard rate function. Establishing its asymptotic properties and specifically its mean square error, reveals that the suggested estimate performs better than its nonparametric counterpart. Detailed instructions are given for calculating the operational characteristics of the estimate, that is, its shape parameter and bandwidth. Finally, its practical performance is illustrated for simulated as well as a real world data example.

Shrinkage-based semiparametric density estimation

Shrinkage estimation is used to develop a semiparametric density estimator as a linear combination of a fully known parametric density function and a nonparametric density estimator. We determine the asymptotic properties of the shrinkage coefficient and of the semiparametric estimator’s integrated squared error. Moreover, we show that the proposed estimation methodology delivers density estimators that are more accurate than nonparametric estimators and that do not require the use of optimal smoothing parameters.

Estimation of Star-Shaped Distributions

Scatter plots of multivariate data sets motivate modeling of star-shaped distributions beyond elliptically contoured ones. We study properties of estimators for the density generator function, the star-generalized radius distribution and the density in a star-shaped distribution model. For the generator function and the star-generalized radius density, we consider a non-parametric kernel-type estimator. This estimator is combined with a parametric estimator for the contours which are assumed to follow a parametric model. Therefore, the semiparametric procedure features the flexibility of nonparametric estimators and the simple estimation and interpretation of parametric estimators. Alternatively, we consider pure parametric estimators for the density. For the semiparametric density estimator, we prove rates of uniform, almost sure convergence which coincide with the corresponding rates of one-dimensional kernel density estimators when excluding the center of the distribution. We show that the standardized density estimator is asymptotically normally distributed. Moreover, the almost sure convergence rate of the estimated distribution function of the star-generalized radius is derived. A particular new two-dimensional distribution class is adapted here to agricultural and financial data sets.

Radar Target Recognition Based on Semiparametric Density Estimation of SLC

Radar Target Recognition Based on Semiparametric Density Estimation of SLC

A new R package for Bayesian estimation of multivariate normal mixtures allowing for selection of the number of components and interval-censored data

An R package mixAK is introduced which implements routines for a semiparametric density estimation through normal mixtures using the Markov chain Monte Carlo (MCMC) methodology. Besides producing the MCMC output, the package computes posterior summary statistics for important characteristics of the fitted distribution or computes and visualizes the posterior predictive density. For the estimated models, penalized expected deviance (PED) and deviance information criterion (DIC) is directly computed which allows for a selection of mixture components. Additionally, multivariate right-, left- and interval-censored observations are allowed. For univariate problems, the reversible jump MCMC algorithm has been implemented and can be used for a joint estimation of the mixture parameters and the number of mixture components. The core MCMC routines have been implemented in C++ and linked to R to ensure a reasonable computational speed. We briefly review the implemented algorithms and illustrate the use of the package on three real examples of different complexity.

Fitting and Predicting Heavy Tailed Insurance Data

Predicting the severity of insurance claims is not difficult when there is correspondence between the observed data and a chosen parametric model. However, for heavy tailed insurance data it is hard to find a parametric model that is sophisticated enough to both predict the tail behavior and fit the body of the data. In the litterature it seems to be a skew attention towards prediction when modeling heavy tailed data and analyzing the fit of the chosen model is given little or no attention. In the paper it is shown that doing so introduces a risk of choosing a distributional model that might predict well but has a poor fit in comparison to other models. By comparing the goodness-of-fit of eligible models it is easier to convince managers and auditors of the superior characteristics of a specific model. In the paper, the prediction and goodness-of-fit of both parametric and semi-parametric density estimators are compared on simulated data as well as on real heavy tailed data from a large general insurer. Also an additional density estimator is introduced which is specially developed for goodness-of-fit and it is argued that this is a favorable choice when modeling heavy tailed data.

Semi-Parametric Density Estimation for Time-Series with Multiplicative Adjustment

In this article, we extend a class of semi-parametric density estimators to time-series context. The asymptotic theory and simulation study are discussed. Theoretical results and numerical comparison show that in the time-series case, the estimators in this class are better than, or at least competitive with, the traditional kernel density estimator in a broad class of densities.

Projection density estimation under a [formula omitted]-sample semiparametric model

An m -sample semiparametric model in which the ratio of m - 1 probability density functions with respect to the mth is of a known parametric form without reference to any parametric model is considered. This model arises naturally from retrospective studies and multinomial logistic regression model. A projection density estimator is constructed by smoothing the increments of the maximum semiparametric empirical likelihood estimator of the underlying distribution function, using the combined data from all the samples. Some asymptotic results on the proposed projection density estimator are established. Connections between our estimator and kernel semiparametric density estimator are pointed out. Some results from simulations and from the analysis of two real data sets are presented.

Local Multiplicative Bias Correction for Asymmetric Kernel Density Estimators

We consider semiparametric asymmetric kernel density estimators when the unknown density has support on [0, ∞). We provide a unifying framework which contains asymmetric kernel versions of several semiparametric density estimators considered previously in the literature. This framework allows us to use popular parametric models in a nonparametric fashion and yields estimators which are robust to misspecification. We further develop a specification test to determine if a density belongs to a particular parametric family. The proposed estimators outperform rival non- and semiparametric estimators in finite samples and are simple to implement. We provide applications to loss data from a large Swiss health insurer and Brazilian income data.

Local multiplicative bias correction for asymmetric kernel density estimators

Abstract We consider semiparametric asymmetric kernel density estimators when the unknown density has support on [ 0 , ∞ ) . We provide a unifying framework which relies on a local multiplicative bias correction, and contains asymmetric kernel versions of several semiparametric density estimators considered previously in the literature. This framework allows us to use popular parametric models in a nonparametric fashion and yields estimators which are robust to misspecification. We further develop a specification test to determine if a density belongs to a particular parametric family. The proposed estimators outperform rival non- and semiparametric estimators in finite samples and are easy to implement. We provide applications to loss data from a large Swiss health insurer and Brazilian income data.

Merging information for a semiparametric projection density estimation

A semiparametric density estimator is proposed under a m-sample density ratio model, which specifies that the ratio of m − 1 probability density functions with respect to the mth is of a known parametric form without reference to any parametric model. This model arises naturally from retrospective studies and multinomial logistic regression model. A projection density estimator is constructed by smoothing the increments of the maximum semiparametric likelihood estimator of the underlying distribution function, using the combined data from all the samples. We also establish some asymptotic results on the proposed projection density estimator. To cite this article: J.-B. Aubin, S. Leoni-Aubin, C. R. Acad. Sci. Paris, Ser. I 344 (2007).

Inequality and Poverty in United States: The Effects of Rising Dispersion of Men's Earnings and Changing Family Behaviour

Using semiparametric density estimation techniques, we analyse the effect of rising dispersion of men's earnings and related changes in family behaviour on increasing inequality in the distribution of family income in the United States. For the period 1969–1989, the growing dispersion of men's earnings and changing family structure can account for most of the rise in family income inequality. By contrast, the increase in labour force participation by women offset this trend. Inequality grew at a slower rate in the 1990s than in earlier decades, largely because of stabilization in the relative earnings of men from low‐income families.

Semiparametric Density Estimators Using Copulas

ABSTRACT In this article we establish semiparametric density estimators by employing the ideas of copulas and density weighting functions. Results on asymptotic normality and uniform strong consistency are derived. The formula for the asymptotic mean square error is used to get the optimal bandwidth. Moreover, applications to several families of copulas are discussed.