Nonparametric Probability Density Estimation Research Articles

Analysis of optimization methods for nonparametric estimation of probability density in large samples

Analysis of optimization methods for nonparametric estimation of probability density in large samples

Analysis of optimization methods for nonparametric estimation of probability density in large volume samples

A method is proposed for selecting the blurriness coefficient of kernel functions for nonparametric estimation of the probability density of a one-dimensional random variable with large volumes of statistical data, for example, obtained by remote sensing of natural objects. In the proposed method for selecting the blurriness coefficient, a regression estimate of the probability density is used. A method for synthesizing a regression probability density estimate is presented. The synthesis of the estimate is based on compression of the initial sample by decomposition of the range of values of a random variable. To decompose the range of values of a random variable, the Heinhold-Gaede rule and the formula for optimal selection of the number of sampling intervals are applied. Two approaches to the selection of the blurriness coefficient of the regression estimation of probability density using the traditional and proposed by the authors optimization methods of nonparametric estimation of probability density are considered. The traditional method of optimizing nonparametric estimation of probability density is based on minimizing its mean square deviation. In the proposed method, the selection of the blurriness coefficients of the kernel functions is based on the conditions of the minimum error of approximation of the regression estimate of the desired probability density. The approximation properties of the regression estimation of probability density using two methods of its optimization are analyzed. The conditions of their competence in estimating the probability densities of random variables with a lognormal distribution law are established. The results obtained allow for development when optimizing a regression estimate of the probability density of a multidimensional random variable.

PDFEstimator: An R Package for Density Estimation and Analysis

This article presents PDFEstimator, an R package for nonparametric probability density estimation and analysis, as both a practical enhancement and alternative to kernel-based estimators. PDFEstimator creates fast, highly accurate, data-driven probability density estimates for continuous random data through an intuitive interface. Excellent results are obtained for a diverse set of data distributions ranging from 10 to $10^6$ samples when invoked with default parameter definitions in the absence of user directives. Additionally, the package contains methods for assessing the quality of any estimate, including robust plotting functions for detailed visualization and trouble-shooting. Usage of PDFEstimator is illustrated through a variety of examples, including comparisons to several kernel density methods.

Research on the target recognition of marine surveillance radar based on ensemble learning

The existence of corner reflector decoy makes the marine surveillance radar to be caught in severe challenges of target recognition. In order to enhance the accuracy of recognition, an ensemble classifier for marine targets is created based on the self-built dataset of high resolution range profile (HRRP). In addition, a confidence evaluation algorithm based on non-parametric estimation of probability density is proposed to reject unknown decoy target outside the database. With taking different interference conditions into consideration respectively, the comparison experiment between ensemble classifier and single classifier is carried out. The results show that the ensemble classifier is significantly better, which has strong robustness to noise as well as rejection ability to unknown target, and the classification accuracy can reach 92.63% under ideal conditions. This paper proves the feasibility of ensemble learning for maritime target recognition, and provides a reliable classification algorithm when the sample information is sufficient.

Nonparametric pattern recognition algorithm for testing a hypothesis of the independence of random variables

A new method for testing a hypothesis of the independence of multidimensional random variables is proposed. The technique under consideration is based on the use of a nonparametric pattern recognition algorithm that meets a maximum likelihood criterion. In contrast to the traditional formulation of the pattern recognition problem, there is no a priori training sample. The initial information is represented by statistical data, which are made up of the values of a multivariate random variable. The distribution laws of random variables in the classes are estimated according to the initial statistical data for the conditions of their dependence and independence. When selecting optimal bandwidths for nonparametric kernel-type probability density estimates, the minimum standard deviation is used as a criterion. Estimates of the probability of pattern recognition error in the classes are calculated. Based on the minimum value of the estimates of the probabilities of pattern recognition errors, a decision is made on the independence or dependence of the random variables. The technique developed is used in the spectral analysis of remote sensing data.

On Time-Dependent Trip Distance Distribution with For-Hire Vehicle Trips in Chicago

For transportation system analysis in a new space dimension with respect to individual trips’ remaining distances, vehicle trips demand has two main components: the departure time and the trip distance. In particular, the trip distance distribution (TDD) is a direct input to the bathtub model in the new space dimension, and is a very important variable to consider in many applications, such as the development of distance-based congestion pricing strategies or mileage tax. For a good understanding of the demand pattern, both the distribution of trip initiation and trip distance should be calibrated from real data. In this paper, it is assumed that the demand pattern can be described by the joint distribution of trip distance and departure time. In other words, TDD is assumed to be time-dependent, and a calibration and validation methodology of the joint probability is proposed, based on log-likelihood maximization and the Kolmogorov–Smirnov test. The calibration method is applied to empirical for-hire vehicle trips in Chicago, and it is concluded that TDD varies more within a day than across weekdays. The hypothesis that TDD follows a negative exponential, log-normal, or Gamma distribution is rejected. However, the best fit is systematically observed for the time-dependent log-normal probability density function. In the future, other trip distributions should be considered and also non-parametric probability density estimation should be explored for a better understanding of the demand pattern.

Adaptive nonparametric estimation of a component density in a two-class mixture model

A two-class mixture model, where the density of one of the components is known, is considered. We address the issue of the nonparametric adaptive estimation of the unknown probability density of the second component. We propose a randomly weighted kernel estimator with a fully data-driven bandwidth selection method, in the spirit of the Goldenshluger and Lepski method. An oracle-type inequality for the pointwise quadratic risk is derived as well as convergence rates over Hölder smoothness classes. The theoretical results are illustrated by numerical simulations.

Multi-index summer flow regime characterization to inform environmental flow contexts: A New England case study

Abstract Summer flow regime is an important determinant of the health and integrity of riverine ecosystems. In particular, summertime low flows and the range of variability impact the extent and connectivity of habitat, production in and composition of ecosystems, as well as water quality. Four streamflow-based metrics are used in the analysis and assessment of summertime hydrologic variability and trends: Lane Index, UK Institute of Hydrology Index, Base Flow Index, and 7-day Minimum Flow Index. In this study, historical streamflow records at 33 locations in New England (USA) were analyzed to understand the nature of trends as well as the extent of changes, individually and collectively, for the four indices. The main contribution of this paper is to develop a comprehensive characterization of the regional flow regime and trends therein. Two complementary statistical analyses applied in this study are: (a) quantile regression-based estimates of the spatial and temporal patterns of changes in flow indices variability over the past six decades and (b) interrelationship between flow indices based on non-parametric probability density estimation approach. The multi-index regional analysis revealed that flow variability (based on the Lane Index) within the summer season has increased significantly, with 25 out of the 33 streamgages showing increases, of which 12 are highly significant. Similar trends are mirrored in the UK Institute of Hydrology Index based analysis. Base Flow Index trends show decreases across 30 of the 33 streamgages at the median level, and 19 are highly significant. Trends are most pronounced at the upper quartile level, with 21 highly significant. Regional analysis of 7-day Minimum Flow Index reveal broad patterns towards increases at the three selected quantile levels ( τ = 0.25 , 0.5 , 0.75 ) with 29, 22, and 19 streamgages showing increases, of which 13, 10, and 6 are highly significant respectively. There is also diversity in trends across quantiles, wherein 17 locations show increases at all quantile levels, while 22 stations showed increases at the median level. The analysis of interrelationships between the four indices revealed inverse variations between Lane Index and 7-day Minimum Flow, and also a diversity of relationships ranging from weakly to strongly nonlinear. As such, our analysis underscores the importance of the use of multiple indices to fully assess variability and change of relevance to management and policy concerns, such as Maine’s water allocation rulemaking.

Modified algorithm for fast bandwidth selection for kernel estimates of multidimensional probability densities

An original technique has been justified for the fast bandwidths selection of kernel functions in a nonparametric estimate of the multidimensional probability density of the Rosenblatt–Parzen type. The proposed method makes it possible to significantly increase the computational efficiency of the optimization procedure for kernel probability density estimates in the conditions of large-volume statistical data in comparison with traditional approaches. The basis of the proposed approach is the analysis of the optimal parameter formula for the bandwidths of a multidimensional kernel probability density estimate. Dependencies between the nonlinear functional on the probability density and its derivatives up to the second order inclusive of the antikurtosis coefficients of random variables are found. The bandwidths for each random variable are represented as the product of an undefined parameter and their mean square deviation. The influence of the error in restoring the established functional dependencies on the approximation properties of the kernel probability density estimation is determined. The obtained results are implemented as a method of synthesis and analysis of a fast bandwidths selection of the kernel estimation of the two-dimensional probability density of independent random variables. This method uses data on the quantitative characteristics of a family of lognormal distribution laws.

Fast Algorithm for Choosing Blur Coefficients in Multidimensional Kernel Probability Density Estimates

A method is proposed for quickly choosing the blur coefficients of kernel functions in a non-parametric estimate of a multidimensional probability density of Rosenblatt–Parzen type. The technique is based on the analysis of the asymptotic properties of a multidimensional probability density estimate. The properties of the fast algorithm for choosing the blur coefficients of a kernel probability density estimate are investigated.

An EM algorithm for fitting a mixture model with symmetric log-concave densities

In this article, we revisit the problem of fitting a mixture model under the assumption that the mixture components are symmetric and log-concave. To this end, we first study the nonparametric maximum likelihood estimation (MLE) of a monotone log-concave probability density. To fit the mixture model, we propose a semiparametric EM (SEM) algorithm, which can be adapted to other semiparametric mixture models. In our numerical experiments, we compare our algorithm to that of Balabdaoui and Doss (2018, Inference for a two-component mixture of symmetric distributions under log-concavity. Bernoulli 24 (2):1053–71) and other mixture models both on simulated and real-world datasets.

A novel whispered speaker identification system based on extreme learning machine

Whispered speech speaker identification system is one of the most demanding efforts in automatic speaker recognition applications. Due to the profound variations between neutral and whispered speech in acoustic characteristics, the performance of conventional speaker identification systems applied on neutral speech degrades drastically when compared to whisper speech. This work presents a novel speaker identification system using whispered speech based on an innovative learning algorithm which is named as extreme learning machine (ELM). The features used in this proposed system are Instantaneous frequency with probability density models. Parametric and nonparametric probability density estimation with ELM was compared with the hybrid parametric and nonparametric probability density estimation with Extreme Learning Machine (HPNP-ELM) for instantaneous frequency modeling. The experimental result shows the significant performance improvement of the proposed whisper speech speaker identification system.

A fast algorithm for choosing the blurring coefficients of nuclear functions in nonparametric estimation of probability density

A fast algorithm for choosing the blurring coefficients of nuclear functions in nonparametric estimation of probability density

Baseband Communication Signal Blind Separation Algorithm Based on Complex Nonparametric Probability Density Estimation

In this paper, a blind separation algorithm based on nonparametric probability density estimation (NPDE) for baseband communication signals is proposed. The NPDE method can precisely estimate the probability density of signals, and its good data-driven property guarantees the adaptability of the proposed algorithm. The NPDE method is extended to complex field to fit the complex characteristics of baseband communication signals. The complex update procedure of demixing matrix corresponding to complex channel matrix and signals is also derived in this paper. The experimental results indicate that the proposed algorithm not only can effectively separate the mixture of baseband communication signals, which are modulated with the same or different modulation schemes, but also has better convergence properties and greater signal interference ratio gains than the contrast algorithms. Moreover, even if the local carrier of receiver is unsynchronized to the carrier of received signals due to carrier offsets, the proposed algorithm can also achieve a valid separation.

Parameter Estimation in Sensor Networks Under Probabilistic Censoring

We consider the problem of estimating an unknown parameter that is observed indirectly by a sensor network. Each of the sensors exhibits probabilistic nonlinear behavior causing censoring (clipping) of the signals at random. The signals then get transmitted over fading wireless channels to the fusion center. We develop three algorithms for estimating the unknown parameter: first is based on Pearson's Method of Moments, which involves the calculation of the statistical moments of the model that can be derive exactly; second is based on the Maximum-Likelihood Estimator , which results in an intractable likelihood function. To evaluate the likelihood function, we develop a novel approximation via a nonparametric probability density estimator that is based on series expansions of the Gram–Charlier family of basis functions; and third is the Marginalized Least Squares in which we solve the least squares problem by marginalizing out the random unknown nuisance parameters of the model. All three algorithms enjoy a low computational complexity, and only involve solving a one-dimensional optimization problem that is simple to implement in practice. We compare the performance of the proposed algorithms under various system configurations and show various tradeoffs between the algorithms as a function of system parameters (e.g., number of sensors, frame length, signal-to-noise ratio, sensing quality etc.).

Event Detection in Sensor Networks With Non-Linear Amplifiers via Mixture Series Expansion

We develop a new algorithm for binary event detection in wireless sensor networks where the sensing process exhibits non-linear behavior causing clipping of the signals at random. We formulate the optimal event detection as a likelihood ratio test under a mixture model, which results in intractable marginal likelihood functions. To evaluate the intractable marginal likelihood functions, we develop a novel approximation via a non-parametric probability density estimator that is based on a series expansion of the Beta-Jacobi family of basis functions. The advantage of our algorithm is that it provides both high detection rate as well as very low computational complexity. In fact, our algorithm only requires the calculation of the first few moments of the model, which can be evaluated offline and is simple to implement in practice. Another feature of our model is that it is general enough to incorporate the probabilistic sensor non-linearities as well as transmission over random wireless channels. Simulation results demonstrate the benefits of using our framework, and that accounting for such practical features significantly improves the detection performance compared with the widely used Gaussian approximation method.

Nonparametric Estimate of a Parzen-Type Probability Density with an Implicitly Specified Form of the Kernel

A nonparametric estimate for the probability density is proposed with better approximation properties than the traditional Rosenblatt–Parzen statistics. The dependences of its properties on the form of the kernel function and on the formulas for the sampling interval for the random quantity are discussed.

An Efficient Reliability-Based Simulation Method for Optimum Laser Peening Treatment

A method is introduced for efficient reliability-based design of laser peening (LP) surface treatment to extend fatigue life of metal components. The method includes nonparametric probability density estimation, surrogate modeling using a new finite element (FE or FEA) approach, and reliability analysis with correlated random variables (RVs). Efficient LP simulation is achieved via a new technique termed single explicit analysis using time-dependent damping (SEATD), which reduces simulation times by a factor of 6. The example study of a three-point bend coupon reveals that fatigue life reliability significantly affects optimal LP design, as 52 laser spots are needed for 99% reliability versus 44 spots for 95%.

Selection of the Optimal Number of Intervals Sampling the Region of Values of a Two-Dimensional Random Variable

We investigate the asymptotic properties of nonparametric estimation of a two-dimensional probability density, whose synthesis involves the decomposition of statistical data. We determine the dependence of the number of two-dimensional sampling intervals on the volume of the original data.

Two-Stage Nonparametric Sequential Estimation of the Directional Density with Complete and Missing Observations

Stein (1945) proposed a two-stage sequential methodology of inference that influenced numerous areas of statistics. In this article, the Stein's methodology is used and expands upon nonparametric estimation of the directional probability density. The aim is to propose a data-driven nonparametric sequential procedure that mimics performance of an oracle that knows smoothness of an estimated density and minimizes the mean stopping time given an assigned mean integrated squared error. For such a setting, using a sequential estimator is the must because smoothness of the density is unknown. It is known that for a general random variable the stated problem has no solution because an estimator cannot perform as well as the minimax oracle. At the same time, this article shows that for the case of directional density, under a mild assumption, there exists a data-driven two-stage sequential procedure that is minimax and adapts to unknown smoothness of an underlying density. Furthermore, we are able to solve the same problem for a setting where some observations in a sample may be lost due to a stochastic missing mechanism. This is a practically important and theoretically interesting problem because for missing data even the above-mentioned oracle cannot solve it without knowing the missing mechanism.