Logspline Density Estimation Research Articles

Penalized logspline density estimation using total variation penalty

We study a penalized logspline density estimation method using a total variation penalty. The B-spline basis is adopted to approximate the logarithm of density functions. Total variation of derivatives of splines is penalized to impart a data-driven knot selection. The proposed estimator is a bona fide density function in the sense that it is positive and integrates to one. We devise an efficient coordinate descent algorithm for implementation and study its convergence property. An oracle inequality of the proposed estimator is established when the quality of fit is measured by the Kullback–Leibler divergence. Based on the oracle inequality, it is proved that the estimator achieves an optimal rate of convergence in the minimax sense. We also propose a logspline method for the bivariate case by adopting the tensor-product B-spline basis and a two-dimensional total variation type penalty. Numerical studies show that the proposed method captures local features without compromising the global smoothness.

An Empirical, Nonparametric Simulator for Multivariate Random Variables with Differing Marginal Densities and Nonlinear Dependence with Hydroclimatic Applications.

Multivariate simulations of a set of random variables are often needed for risk analysis. Given a historical data set, the goal is to develop simulations that reproduce the dependence structure in that data set so that the risk of potentially correlated factors can be evaluated. A nonparametric, copula-based simulation approach is developed and exemplified. It can be applied to multiple variables or to spatial fields with arbitrary dependence structures and marginal densities. The nonparametric simulator uses logspline density estimation in the univariate setting, together with a sampling strategy to reproduce dependence across variables or spatial instances, through a nonparametric numerical approximation of the underlying copula function. The multivariate data vectors are assumed to be independent and identically distributed. A synthetic example is provided to illustrate the method, followed by an application to the risk of livestock losses in Mongolia.

Construction of joint confidence regions for the optimal true class fractions of Receiver Operating Characteristic (ROC) surfaces and manifolds.

The three-class approach is used for progressive disorders when clinicians and researchers want to diagnose or classify subjects as members of one of three ordered categories based on a continuous diagnostic marker. The decision thresholds or optimal cut-off points required for this classification are often chosen to maximize the generalized Youden index (Nakas etal., Stat Med 2013; 32: 995-1003). The effectiveness of these chosen cut-off points can be evaluated by estimating their corresponding true class fractions and their associated confidence regions. Recently, in the two-class case, parametric and non-parametric methods were investigated for the construction of confidence regions for the pair of the Youden-index-based optimal sensitivity and specificity fractions that can take into account the correlation introduced between sensitivity and specificity when the optimal cut-off point is estimated from the data (Bantis etal., Biomet 2014; 70: 212-223). A parametric approach based on the Box-Cox transformation to normality often works well while for markers having more complex distributions a non-parametric procedure using logspline density estimation can be used instead. The true class fractions that correspond to the optimal cut-off points estimated by the generalized Youden index are correlated similarly to the two-class case. In this article, we generalize these methods to the three- and to the general k-class case which involves the classification of subjects into three or more ordered categories, where ROC surface or ROC manifold methodology, respectively, is typically employed for the evaluation of the discriminatory capacity of a diagnostic marker. We obtain three- and multi-dimensional joint confidence regions for the optimal true class fractions. We illustrate this with an application to the Trail Making Test Part A that has been used to characterize cognitive impairment in patients with Parkinson's disease.

A Logspline Estimation for a Linear Regression Model with an Interval-Censored Continuous Covariate

The study of a linear regression model with an interval-censored covariate, which was motivated by an acquired immunodeficiency syndrome (AIDS) clinical trial, was first proposed by Gómez et al. They developed a likelihood approach, together with a two-step conditional algorithm, to estimate the regression coefficients in the model. However, their method is inapplicable when the interval-censored covariate is continuous. In this article, we propose a novel and fast method to treat the continuous interval-censored covariate. By using logspline density estimation, we impute the interval-censored covariate with a conditional expectation. Then, the ordinary least-squares method is applied to the linear regression model with the imputed covariate. To assess the performance of the proposed method, we compare our imputation with the midpoint imputation and the semiparametric hierarchical method via simulations. Furthermore, an application to the AIDS clinical trial is presented.

Construction of confidence regions in the ROC space after the estimation of the optimal Youden index‐based cut‐off point

After establishing the utility of a continuous diagnostic marker investigators will typically address the question of determining a cut-off point which will be used for diagnostic purposes in clinical decision making. The most commonly used optimality criterion for cut-off point selection in the context of ROC curve analysis is the maximum of the Youden index. The pair of sensitivity and specificity proportions that correspond to the Youden index-based cut-off point characterize the performance of the diagnostic marker. Confidence intervals for sensitivity and specificity are routinely estimated based on the assumption that sensitivity and specificity are independent binomial proportions as they arise from the independent populations of diseased and healthy subjects, respectively. The Youden index-based cut-off point is estimated from the data and as such the resulting sensitivity and specificity proportions are in fact correlated. This correlation needs to be taken into account in order to calculate confidence intervals that result in the anticipated coverage. In this article we study parametric and non-parametric approaches for the construction of confidence intervals for the pair of sensitivity and specificity proportions that correspond to the Youden index-based optimal cut-off point. These approaches result in the anticipated coverage under different scenarios for the distributions of the healthy and diseased subjects. We find that a parametric approach based on a Box-Cox transformation to normality often works well. For biomarkers following more complex distributions a non-parametric procedure using logspline density estimation can be used.

Learned-loss boosting

This paper considers a problem of jointly estimating a regression function and the distribution of residuals when both are specified non-parametrically. We present a joint penalized optimization criterion that combines log-spline density estimation with spline-based regression methods. We also examine the use of boosting methodology to estimate a regression function over a high dimensional covariate space. We demonstrate that our method has a robustification effect, and show its usefulness in diagnosing problems in data. We illustrate our methods with practical examples when likelihood is an appropriate evaluation criterion.

Indirect estimation of porcine parvovirus maternal immunity decay in free-living wild boar ( Sus scrofa) piglets by capture–recapture data

Linear mixed regression and logspline density estimation were performed to estimate the survival curve and half life of passively acquired antibodies against porcine parvovirus (PPV) in 44 wild boar piglets captured in the Northern Apennines, Italy. One piglet had no detectable maternal antibodies at 2.5 months post partum and no antibodies were detected in any of the remaining piglets by 4 months of age. Fitted survival curves indicated that maternal antibodies were undetectable from 2.5 to 6 months of age, with a median value at 3 months and a low probability of persistent maternal antibodies after 4 months of age. The estimated half life was 23 days (95% confidence interval 22–26 days). The results agree with previous data for decay of maternally acquired antibodies against PPV in the domestic pig and indicate the value of capture–recapture analysis for the estimation of infection parameters in free-living animals.

Comparison of Parametric and Bootstrap Approaches to Obtaining Confidence Intervals for Logspline Density Estimation

In earlier articles, we developed an automated methodology for using cubic splines with tail linear constraints to model the logarithm of a univariate density function. This methodology was subsequently modified so that the knots were determined by stepwise addition-deletion and the remaining coefficients were determined by maximum likelihood estimation. An alternative approach, referred to as the free knot spline procedure, is to use the maximum likelihood method to estimate the knot locations as well as the remaining coefficients. This article compares various approaches to constructing confidence intervals for logspline density estimates, for both the stepwise procedure and the free knot procedure. It is concluded that a variation of the bootstrap, in which only a limited number of bootstrap simulations are used to estimate standard errors that are combined with standard normal quantiles, seems to perform the best, especially when coverages and computing time are both taken into account.

Spline Adaptation in Extended Linear Models (with comments and a rejoinder by the authors

In many statistical applications, nonparametric modeling can provide insights into the features of a dataset that are not obtainable by other means. One successful approach involves the use of (univariate or multivariate) spline spaces. As a class, these methods have inherited much from classical tools for parametric modeling. For example, stepwise variable selection with spline basis terms is a simple scheme for locating knots (breakpoints) in regions where the data exhibit strong, local features. Similarly, candidate knot configurations (generated by this or some other search technique), are routinely evaluated with traditional selection criteria like AIC or BIC. In short, strategies typically applied in parametric model selection have proved useful in constructing flexible, low-dimensional models for nonparametric problems. Until recently, greedy, stepwise procedures were most frequently suggested in the literature. Research into Bayesian variable selection, however, has given rise to a number of new spline-based methods that primarily rely on some form of Markov chain Monte Carlo to identify promising knot locations. In this paper, we consider various alternatives to greedy, deterministic schemes, and present a Bayesian framework for studying adaptation in the context of an extended linear model (ELM). Our major test cases are Logspline density estimation and (bivariate) Triogram regression models. We selected these because they illustrate a number of computational and methodological issues concerning model adaptation that arise in ELMs.

Logspline density estimation for binned data

In this paper we consider logspline density estimation for binned data. Rates of convergence are established when the log-density function is assumed to be in a Besov space. An algorithm involving a procedure similar to maximum likelihood, stepwise knot addition, and stepwise knot deletion is proposed for the estimation of the density function based upon binned data. Numerical examples are used to show the finite-sample performance of inference based on the logspline density estimation.

Logspline Deconvolution in Besov Space

ABSTRACT. In this paper we consider logspline density estimation for random variables which are contaminated with random noise. In the logspline density estimation for data without noise, the logarithm of an unknown density function is estimated by a polynomial spline, the unknown parameters of which are given by maximum likelihood. When noise is present, B‐splines and the Fourier inversion formula are used to construct the logspline density estimator of the unknown density function. Rates of convergence are established when the log‐density function is assumed to be in a Besov space. It is shown that convergence rates depend on the smoothness of the density function and the decay rate of the characteristic function of the noise. Simulated data are used to show the finite‐sample performance of inference based on the logspline density estimation.

Logspline Density Estimation under Censoring and Truncation

ABSTRACT. In this paper we consider logspline density estimation for data that may be left‐truncated or right‐censored. For randomly left‐truncated and right‐censored data the product‐limit estimator is known to be a consistent estimator of the survivor function, having a faster rate of convergence than many density estimators. The product‐limit estimator and B‐splines are used to construct the logspline density estimate for possibly censored or truncated data. Rates of convergence are established when the log‐density function is assumed to be in a Besov space. An algorithm involving a procedure similar to maximum likelihood, stepwise knot addition, and stepwise knot deletion is proposed for the estimation of the density function based upon sample data. Numerical examples are used to show the finite‐sample performance of inference based on the logspline density estimation.

Convergence Rates for Logspline Tomography

We consider bivariate logspline density estimation for tomography data. In the usual logspline density estimation for bivariate data, the logarithm of the unknown density function is estimated by tensor product splines, the unknown parameters of which are given by maximum likelihood. In this paper we use tensor product B-splines and the projection-slice theorem to construct the logspline density estimators for tomography data. Rates of convergence are established for log-density functions assumed to belong to a Besov space.

Bivariate B-splines for tensor logspline density estimation

Tensor logspline density estimation based on bivariate B-splines is considered for estimating unknown bivariate density functions. It is useful for describing the graphical structure of two continuous variables. Several numerical properties of tensor-product B-splines are incorporated into the computation of the density estimates. A simple adaptive knot-selection rule, which involves stepwise knot deletion and BIC, is used to select the final model. Examples using both simulated and real data are used to illustrate the performance of this method.

Logspline Density Estimation for Censored Data

Abstract Logspline density estimation is developed for data that may be right censored, left censored, or interval censored. A fully automatic method, which involves the maximum likelihood method and may involve stepwise knot deletion and either the Akaike information criterion (AIC) or Bayesian information criterion (BIC), is used to determine the estimate. In solving the maximum likelihood equations, the Newton–Raphson method is augmented by occasional searches in the direction of steepest ascent. Also, a user interface based on S is described for obtaining estimates of the density function, distribution function, and quantile function and for generating a random sample from the fitted distribution.

A study of logspline density estimation

A method of estimating an unknown density function ƒ based on sample data is studied. Our approach is to use maximum likelihood etimation to estimate log(ƒ) by a function s from a space of cubic splines that have a finite number of prespecified knots and are linear in the tails. The knots are placed at selected order statistics of the sample data. The number of knots can be determined either by a simple rule or by minimizing a variant of AIC. Examples using both simulated and real data show that the method works well both in obtaining smooth estimates and in picking up small details. The method is fully automatic and can easily be extended to yield estimates and confidence bounds for quantiles.