Estimation Of Probability Density Function Research Articles

Relationship Between Kendall's tau Correlation and Mutual Information

Mutual information (MI) can be viewed as a measure of multivariate association in a random vector. However, the estimation of MI is difficult since the estimation of the joint probability density function (PDF) of non Gaussian distributed data is a hard problem. Copula function is an appropriate tool for estimating MI since the joint probability density function ofrandom variables can be expressed as the product of the associated copula density function and marginal PDF’s. With a little search, we find that the proposed copulas-based mutual information is much more accurate than conventional methods such as the joint histogram and Parzen window-based MI. In this paper, by using the copulas-based method, we compute MI forsome family of bivariate distribution functions and study the relationship between Kendall’s tau correlation and MI of bivariate distributions. Finally, using a real dataset, we illustrate the efficiency of this approach.

Multivariate generalised gamma kernel density estimators and application to non-negative data

This paper proposes a classical multivariate generalised gamma (GG) kernel estimator for probability density function (pdf) estimation in the context of multivariate nonnegative data. Then, we show that the multiplicative bias correction (MBC) techniques can be applied for multivariate GG kernel density estimator as in Funke and Kawka (2015). Some properties (bias, variance and mean integrated squared error) of the corresponding estimators are also provided. The choice of the vector of bandwidths is investigated by adopting the popular cross-validation technique. Finally, the performances of the classical and MBC estimator based on the family of GG kernels are illustrated by a simulation study and real data.

Multivariate generalised gamma kernel density estimators and application to non-negative data

This paper proposes a classical multivariate generalised gamma (GG) kernel estimator for probability density function (pdf) estimation in the context of multivariate nonnegative data. Then, we show that the multiplicative bias correction (MBC) techniques can be applied for multivariate GG kernel density estimator as in Funke and Kawka (2015). Some properties (bias, variance and mean integrated squared error) of the corresponding estimators are also provided. The choice of the vector of bandwidths is investigated by adopting the popular cross-validation technique. Finally, the performances of the classical and MBC estimator based on the family of GG kernels are illustrated by a simulation study and real data.

An Application of New Method to Obtain Probability Density Function of Solution of Stochastic Differential Equations

Abstract In this study, a new method to obtain approximate probability density function (pdf) of random variable of solution of stochastic differential equations (SDEs) by using generalized entropy optimization methods (GEOM) is developed. By starting given statistical data and Euler–Maruyama (EM) method approximating SDE are constructed several trajectories of SDEs. The constructed trajectories allow to obtain random variable according to the fixed time. An application of the newly developed method includes SDE model fitting on weekly closing prices of Honda Motor Company stock data between 02 July 2018 and 25 March 2019.

Bayesian Fill Volume Estimation Based on Point Level Sensor Signals

In dry bulk and fluid processing, the composites are usually stored in hoppers, tanks, or other containers. Due to the economic advantages, binary point level sensors, which detect fill level exceeding, are widely used for process monitoring and control. In this paper, we propose different filters for estimating the probability distribution of the fill volume based on a time-variant measurement distribution and a stochastic physical model with white process noise. A filter based on the model prediction with separated measurement update and two Bayesian particle filters are proposed and compared with a simulated ground truth. The performance measures are the root-mean-square error, the precision of the 95 % and 75 % credible intervals, and the average value of the estimated probability density function at the simulated fill volumes.

A novel estimation method for failure-probability-based-sensitivity by conditional probability theorem

By the average absolute difference between the unconditional failure probability and the conditional one on fixing an input at its realization, the failure-probability-based-sensitivity (FP-S) is defined to quantify the effect of the fixed input on the failure probability, which provides important information for reliability-based design optimization of the structure. Among the estimation methods for FP-S, the Bayes theorem-based methods are competitive, but the conditional probability density function (PDF) should be estimated in this type method. To alleviate the computational complexity of estimating conditional PDF, a novel FP-S estimation method is proposed by use of the conditional probability theorem. In the proposed method, the conditional failure probability on fixing the input at its realization is approximated by the conditional failure probability on fixing the input in a small interval, in which the conditional probability theorem of the random event can be used to transform FP-S as estimations of a series of probabilities, and they can be simultaneously completed by a numerical simulation for estimating the unconditional failure probability. For ensuring the precision of the approximation introduced by replacing the realization with the small interval, a selection strategy for the small interval is proposed. Comparing with the competitive Bayes theorem-based estimation for FP-S, the proposed method replaces the conditional PDF estimation with the conditional probability estimation, which greatly reduces the computational complexity and improves the accuracy of the FP-S estimation. By combining with the adaptive kriging surrogate model, the efficiency of the proposed method can be drastically improved, and the presented examples demonstrate the efficiency and accuracy of the proposed method.

Reformative translation model for probability density functions of amplitude processes of crosswind-excited super-tall buildings

At the vicinity of vortex lock-in wind speed, the nonlinear aerodynamic damping effect of super-tall buildings is significant, which can greatly promote the surge of vortex-induced vibration in the crosswind direction, where the crosswind response characterized by harmonic amplitude shows narrow-band hardening non-Gaussian characteristic with the kurtosis well below 3, and the corresponding probability distribution of amplitude process distinctly differs from that of typical random buffeting response. Although the moment-based Hermite translation model has been widely used for estimating the extreme value distribution of non-Gaussian process, it fails to represent the probability distribution of hardening non-Gaussian amplitude process, notably for the response with a kurtosis close to 1.5. In this study, a new translation model based on orthogonal expansion of random processes is developed for obtaining the non-Gaussian amplitude process from an underlying Gaussian amplitude process, and the probability density function of the non-Gaussian amplitude process is derived by mapping the cumulative distribution function. The coefficients of translation model are determined by minimizing the errors between the estimated probability density functions and target values through nonlinear optimization, and the closed-form semi-empirical formulations, which connect the model coefficients with response kurtosis, are also proposed using least-square curve fitting. Moreover, the effectiveness and monotonicity of the proposed translation model are examined. This model can be readily incorporated into the extreme value analysis of crosswind response and facilitate the evaluation of wind-induced fatigue of super-tall buildings.

Elucidating the Auxiliary Particle Filter via Multiple Importance Sampling [Lecture Notes

Sequential Monte Carlo methods, also known as particle filtering, have seen an explosion of development both in theory and applications. The publication of [1] sparked huge interest in the area of sequential signal processing, particularly in sequential filtering. Ever since, the number of publications in which particle filtering plays a prominent role has continued to grow. An early reference of development is [2] and later tutorials include [3]-[9]. With particle filtering, we estimate probability density functions (pdfs) of interest by probability mass functions, whose masses are placed at randomly chosen locations (particles) and whose weights are assigned to the particles. The particle filter (PF) proposed in [1] is often called the bootstrap PF (BPF), and although it is not optimal, it is the most often used filter by practitioners. A filter that became also popular is known as the auxiliary PF (APF) and was proposed in [10]. With the APF, the objective is to generate better particles at each time step compared to those generated with the BPF, thereby improving filtering accuracy. In this article, we derive the APF from a new perspective, one based on interpreting the APF from the multiple importance sampling (MIS) paradigm. The derivation also shows its relationship with the BPF.

Prior distribution-based statistical active contour model

Employing prior information can greatly improve the segmentation result of many image segmentation problems. For example, a commonly used prior information is the shape of the object. In this paper, we introduce a different kind of prior information called the prior distribution. On the basis of non-parametric statistical active contour model, we add prior distribution energy to build a novel prior active contour model. During the convergence of contour curve, distribution difference between the inside and outside of the active contour is maximized while the distribution difference between the inside/outside of contour and the prior object/background is minimized. Furthermore, in order to improve the computation speed, a method to accelerate the computation speed is also proposed, which significantly relieves the burden of estimating probability density functions. As the experimental results suggest, satisfactory effects can be achieved in the segmentation of synthetic images and natural images via the our algorithm. Compared with the traditional non-parametric statistical active contour model without prior information, our method achieves a distinct improvement in both accuracy and computation efficiency.

Secrecy Analysis in DF Relay Over Generalized-$K$ Fading Channels

In this paper, we analyze the secrecy performance of the decode-and-forward (DF) relay system in generalized- $K$ fading channels. In a typical four-node communications model, a source ( $S$ ) sends confidential information to a destination ( $D$ ) via a relay ( $R$ ) using DF strategy in two time slots, while an eavesdropper ( $E$ ) wants to overhear the information from $S$ to $D$ over generalized- $K$ fading channels. To be more realistic, we assume that $E$ can receive the signals of two time slots, and there is no direct link between $S$ and $D$ because of heavy fading. Based on those assumptions, we derive closed-form expressions for the secrecy outage probability (SOP) and ergodic secrecy capacity (ESC) by using a tight approximate probability density function of the generalized- $K$ model. Furthermore, asymptotic expressions for the SOP and ESC are also derived in the high signal-to-noise ratio region, not only because we can get some insights about SOP and ESC, but also because expressions for SOP and ESC can be simplified significantly. The single relay system is subsequently extended into a multi-relay system, where the asymptotic SOP analysis of three proposed relay selection strategies is investigated. Moreover, the security-reliability tradeoff analysis in the multi-relay system is also presented given that $S$ adopts a constant code rate. Finally, the Monte-Carlo simulation is used to demonstrate the accuracy of the derived closed-form expressions.

Tree Copula Theory Based Fusion And Compressive Sensing For Activity Detection using Multi-Modal Data

Activity detection based on likelihood ratio in the presence of high dimensional multimodal data acts as a challenging problem as the estimation of joint probability density functions (pdfs) with intermodal dependence is tedious. The existing method with above expectations fails due to poor performance in the presence of strongly dependent data. This paper proposes a Compressive Sensing Based Detection method in the Multi-sensor signal using the deep learning method. The proposed Tree copula- Grasshopper optimization based Deep Convolutional Neural Network (TC-GO based DCNN) detection method comprises of three main steps, such as compressive sensing, fusion and detection. The signals are initially collected from the sensors in order to subject them under tensor based compressive sensing. The compressed signals are then fused together using tree copula theory, and the parameters are estimated with the Grasshopper optimization algorithm (GOA). The activity detection is finally performed using DCNN, which is trained with the Stochastic Gradient Descent (SGD) Optimizer. The performance of the proposed method is evaluated based on the evaluation metrics, such as probability of detection and probability of false alarm. The highest probability of detection and least probability of false alarm are obtained as 0.9083, and 0.0959, respectively using the proposed method that shows the effectiveness of the proposed method in activity detection.

Fast and simple dataset selection for machine learning

Abstract The task of data reduction is discussed and a novel selection approach which allows to control the optimal point distribution of the selected data subset is proposed. The proposed approach utilizes the estimation of probability density functions (pdfs). Due to its structure, the new method is capable of selecting a subset either by approximating the pdf of the original dataset or by approximating an arbitrary, desired target pdf. The new strategy evaluates the estimated pdfs solely on the selected data points, resulting in a simple and efficient algorithm with low computational and memory demand. The performance of the new approach is investigated for two different scenarios. For representative subset selection of a dataset, the new approach is compared to a recently proposed, more complex method and shows comparable results. For the demonstration of the capability of matching a target pdf, a uniform distribution is chosen as an example. Here the new method is compared to strategies for space-filling design of experiments and shows convincing results.

New approximate and asymptotic closed-form expressions for the outage probability and the average BER of MIMO-FSO system with MRC diversity technique over Gamma–Gamma fading channels with generalized pointing errors

This paper studies the outage probability (OP) and the average bit-error rate (BER) of multiple-input multiple-output (MIMO) free-space optical (FSO) system with maximal ratio combining (MRC) diversity technique over Gamma–Gamma (GG) fading channels with generalized pointing errors modeled by the Beckmann distribution. A new approximate closed-form probability density function (PDF) at high signal-to-noise ratio (SNR) for MRC scheme is derived by inverse Laplace transformation method. Based on the developed PDF, new approximate and asymptotic closed-form expressions for the OP and the average BER are derived, respectively. Then, the OP and the average BER performance of the MIMO-FSO system with MRC scheme at high SNR are analyzed with different parameters, including number of transmitters and receivers, receiver aperture sizes, nonzero boresight, and different jitters. Finally, numerical results and Monte Carlo simulations corroborate the accuracy of the derived approximate and asymptotic expressions over a wide range of SNR.

Spatial and Temporal Variability of Ambient Underwater Sound in the Baltic Sea

During last decades, anthropogenic underwater sound and its chronic impact on marine species have been recognised as an environmental protection challenge. At the same time, studies on the spatial and temporal variability of ambient sound, and how it is affected by biotic, abiotic and anthropogenic factors are lacking. This paper presents analysis of a large-scale and long-term underwater sound monitoring in the Baltic Sea. Throughout the year 2014, sound was monitored in 36 Baltic Sea locations. Selected locations covered different natural conditions and ship traffic intensities. The 63 Hz, 125 Hz and 2 kHz one-third octave band sound pressure levels were calculated and analysed. The levels varied significantly from one monitoring location to another. The annual median sound pressure level of the quietest and the loudest location differed almost 50 dB in the 63 Hz one-third octave band. Largest difference in the monthly medians was 15 dB in 63 Hz one-third octave band. The same monitoring locations annual estimated probability density functions for two yearly periods show strong similarity. The data variability grows as the averaging time period is reduced. Maritime traffic elevates the ambient sound levels in many areas of the Baltic Sea during extensive time periods.

Accelerating MCMC via Kriging-based adaptive independent proposals and delayed rejection

Markov Chain Monte Carlo (MCMC) simulation has a considerable computational burden when the target probability density function (PDF) evaluation involves a black-box, potentially computationally-expensive, numerical model. A novel framework to accelerate MCMC is developed here for such applications. It leverages a Kriging surrogate model approximation to the target PDF to improve computational efficiency, while preserves convergence properties to the exact target PDF, avoiding potential accuracy problems introduced through the surrogate model error. The approach relies on the delayed-rejection (DR) scheme and combines the rapid exploration characteristics of global (independent) proposals with the local search robustness of random walk proposals under the MCMC setting. The global proposal is chosen as the Kriging-based target PDF approximation. This proposal may resemble, depending on the surrogate model error, the actual target PDF very well, and therefore can provide significant computational benefits, including fast convergence of the Markov chain to its stationary distribution and, more importantly, low correlation between samples. At each MCMC step a candidate sample is generated from the independent proposal. If rejected, DR allows an extra random walk around the current sample. The DR step guarantees convergence to the actual target PDF, circumventing robustness problems that may arise when the Kriging-based independent proposal offers a poor approximation to (i.e., underestimates) the actual target PDF. The overall computational efficiency is further improved through an adaptive updating of the Kriging surrogate model during the MCMC sampling phase by extracting information from candidate samples whose inclusion in the training database can substantially enhance Kriging’s accuracy. The computational efficiency and robustness of the established algorithm, termed Adaptive Kriging Delayed Rejection Adaptive Metropolis algorithm(AK-DRAM), are demonstrated in two analytical benchmark problems and two engineering problems focusing on conditional failure sample simulation and Bayesian inference.

An approximate technique for determining in closed form the response transition probability density function of diverse nonlinear/hysteretic oscillators

An approximate analytical technique is developed for determining, in closed form, the transition probability density function (PDF) of a general class of first-order stochastic differential equations (SDEs) with nonlinearities both in the drift and in the diffusion coefficients. Specifically, first, resorting to the Wiener path integral most probable path approximation and utilizing the Cauchy–Schwarz inequality yields a closed-form expression for the system response PDF, at practically zero computational cost. Next, the accuracy of this approximation is enhanced by proposing a more general PDF form with additional parameters to be determined. This is done by relying on the associated Fokker–Planck operator to formulate and solve an error minimization problem. Besides the mathematical merit of the derived closed-form approximate PDFs, an additional significant advantage of the technique relates to the fact that it can be readily coupled with a stochastic averaging treatment of second-order SDEs governing the dynamics of diverse stochastically excited nonlinear/hysteretic oscillators. In this regard, it is shown that the technique is capable of determining approximately the response amplitude transition PDF of a wide range of nonlinear oscillators, including hysteretic systems following the Preisach versatile modeling. Several numerical examples are considered for demonstrating the reliability and computational efficiency of the technique. Comparisons with pertinent Monte Carlo simulation data are provided as well.

Estimation of Probability Distribution and Its Application in Bayesian Classification and Maximum Likelihood Regression

Nonparametric estimation of cumulative distribution function and probability density function of continuous random variables is a basic and central problem in probability theory and statistics. Although many methods such as kernel density estimation have been presented, it is still quite a challenging problem to be addressed to researchers. In this paper, we proposed a new method of spline regression, in which the spline function could consist of totally different types of functions for each segment with the result of Monte Carlo simulation. Based on the new spline regression, a new method to estimate the distribution and density function was provided, which showed significant advantages over the existing methods in the numerical experiments. Finally, the density function estimation of high dimensional random variables was discussed. It has shown the potential to apply the method in classification and regression models.

A new fractional moment equation method for the response prediction of nonlinear stochastic systems

Moment equation method is commonly used in the analysis of nonlinear stochastic systems. However, in some cases, this method may lead to poor accuracy, or even invalid results, because only limited number of moments of the response is available. In order to overcome this limitation, this paper develops a new fractional moment equation methodology. The new method involves the derivation of fractional moment equation of nonlinear systems and the approximation of probability density function (PDF) of response by means of a novel fractional moment closure scheme. Benefitting from the valuable features of fractional moments, i.e., few number of fractional moments contain large amount of statistical information about the variable, the developed method achieves more accurate PDF estimation of the response when compared with the present methods, especially for nonlinear systems with multiple equilibria. The effectiveness of the new method is finally demonstrated by a Duffing and a bistable Duffing oscillator that is subjected to Gaussian white noise. The results are compared with the Gaussian closure approximation and exact solution.

Asymptotic capacity analysis under different adaptive transmission for TAS/MRC MIMO scheme subject to Weibull fading channels

This paper investigates a unified average channel capacity (ACC) analysis under different adaptive transmission of multiple-input multiple-output (MIMO) system, combining single transmit antenna selection (TAS) and maximal-ratio combining (MRC) receiver operating under independent flat Weibull multipath fading channels (WFCs). Based on a tight approximate probability density function expression of the signal-to-noise ratio (SNR) at the considered receiver output, we derive new accurate closed-form expressions of ACC for TAS/MRC system employing several rate adaptation techniques, namely optimal rate adaptation, optimal simultaneous power and rate adaptation, channel inversion with fixed rate (CIFR), truncated CIFR (TCIFR), and continuous power TCIFR (CTCIFR). Further, asymptotic expressions, in both high and low-SNR regimes, are investigated. The results show high accuracy for significant values of Lt×Lr TAS/MRC system. Indeed, the smaller Lt×Lr, the better the PDF’s accuracy, therefore, the better are the capacities. Numerical outcomes have been assessed by utilizing Mathematica and Matlab Software and corroborated via Monte-Carlo simulations.

Dynamic state estimation of power systems using intelligent particle filtering based on ant colony optimisation for continuous domains

In this study, a dynamic state estimation (DSE) approach is proposed for power systems based on particle filter (PF) and ant colony optimisation for continuous domains (ACOR). Usually, the Kalman-based estimators with Gaussian noise assumption are utilised for DSE. However, this assumption is questionable for real power system data. Although PF methods offer a potential solution for this issue, the basic PF algorithm is time-consuming and suffers from sample impoverishment problem and degeneracy of the propagated samples. In this study, the search capability of ACOR is utilised as an adaptive probability density function estimator to overcome these shortcomings and reduce the required particle numbers. The proposed approach minimises the computational effort by reducing the required particle numbers. The ninth-order model of the synchronous generator has been applied in this study. The performance of the proposed method is investigated through a two-area-four-machine test system as well as the IEEE 39-bus (New England) test system and it is compared with several advanced Kalman filter-based and PF-based approaches. The simulation results obtained for different case studies demonstrate the effectiveness and robustness of the proposed method against noises, abrupt state changes, and gross measurement errors.