Probability Density Estimation Research Articles

Deep learning-based ensemble forecast and predictability analysis of the Kuroshio intrusion into the South China Sea

Abstract The Kuroshio intrusion (KI) into the South China Sea (SCS) significantly affects the environment, ecology, and climate change of the SCS. However, due to the nonlinearity of KI, its numerical prediction often requires large ensemble size to measure prediction uncertainty. The huge computational costs of large numbers of members and high-resolution numerical models pose significant challenges for KI prediction. We, therefore, construct a Kuroshio ensemble deep learning prediction system (KurNet) through taking different values of parameters to predict KI paths because the deep learning models have high prediction skills and low computational cost. The KurNet containing 64 ensemble members can not only output ensemble mean forecast result of the Kuroshio path, but also estimate probability density functions for the path types. The KurNet illustrates a high predictive ability for the KI with the mean classification accuracy of 71.9% and root mean square error of 0.913 on the testing set, which is superior to the single control prediction by ∼1.0–2.9%, although the control prediction model is selected as one of the ensemble members with the best predictive ability on the validation set. Furthermore, the predictability analysis of 10 KI events indicates that when the lead time is 3 days, the most important areas are in the east of Luzon Island due to the upstream Kuroshio transport. As the lead time increases, the most important area is in the Luzon Strait due to the eddy activity. Observing system simulation experiments reveal that the KI forecast skill can be enhanced by ∼12–18%, when uncertainties of the input data in these important regions are removed.

Recognition method for biomimetic camouflage communication signal imitating cetacean click in underwater multipath channels

Focusing on the need of biomimetic camouflage communication signal recognition in underwater multipath channels, we propose a recognition method for biomimetic camouflage click communication train (BCCCT) based on time delay difference (TDD) coding in multipath channels. For the problem of multipath interference, we perform multipath determination on the received signal by autocorrelation, filter relevant peak points, determine the time delays between multipaths, and remove the multipath interference signals according to the determined time delays. A click extraction method based on the double sliding window is proposed, which locates the boundary of click by signal envelope. By analyzing the distribution characteristics of the time interval of adjacent clicks (TIAC) of BCCCT and real click train (RCT), we model the TIACs based on Gaussian kernel probability density estimation (PDE), and recognize the BCCCTs by the Convolutional Neural Networks (CNN), which takes the PDE curve as the system input. The numerical simulations and a lake experiment are carried out to measure the effectiveness of the recognition method proposed under different signal-to-noise ratios (SNR), coding parameters, and channels. The recognition accuracy can achieve 80 % at 0 dB in the constructed channels. Moreover, in the real underwater communication environment, the recognition accuracy can achieve 95 %.

Deep Hawkes process for high-frequency market making

AbstractHigh-frequency market making is a liquidity-providing trading strategy that simultaneously generates many bids and asks for a security at ultra-low latency while maintaining a relatively neutral position. The strategy makes a profit from the bid-ask spread for every buy and sell transaction, against the risk of adverse selection, uncertain execution and inventory risk. We design realistic simulations of limit order markets and develop a high-frequency market making strategy in which agents process order book information to post the optimal price, order type and execution time. By introducing the Deep Hawkes process to the high-frequency market making strategy, we allow a feedback loop to be created between order arrival and the state of the limit order book, together with self- and cross-excitation effects. Our high-frequency market making strategy accounts for the cancellation of orders that influence order queue position, profitability, bid-ask spread and the value of the order. The experimental results show that our trading agent outperforms the baseline strategy, which uses a probability density estimate of the fundamental price. We investigate the effect of cancellations on market quality and the agent’s profitability. We validate how closely the simulation framework approximates reality by reproducing stylised facts from the empirical analysis of the simulated order book data.

Coping with the Inequity and Inefficiency of the H-Index: A Cross-Disciplinary Empirical Analysis

This paper measures two main inefficiency features (many publications other than articles; many co-authors’ reciprocal citations) and two main inequity features (more co-authors in some disciplines; more citations for authors with more experience). It constructs a representative dataset based on a cross-disciplinary balanced sample (10,000 authors with at least one publication indexed in Scopus from 2006 to 2015). It estimates to what extent four additional improvements of the H-index as top-down regulations (∆Hh = Hh − Hh+1 from H1 = based on publications to H5 = net per-capita per-year based on articles) account for inefficiency and inequity across twenty-five disciplines and four subjects. Linear regressions and ANOVA results show that the single improvements of the H-index considerably and decreasingly explain the inefficiency and inequity features but make these vaguely comparable across disciplines and subjects, while the overall improvement of the H-index (H1–H5) marginally explains these features but make disciplines and subjects clearly comparable, to a greater extent across subjects than disciplines. Fitting a Gamma distribution to H5 for each discipline and subject by maximum likelihood shows that the estimated probability densities and the percentages of authors characterised by H5 ≥ 1 to H5 ≥ 3 are different across disciplines but similar across subjects.

BOUNDARY BETWEEN THE TIMANIAN AND URALIAN TECTONIC CYCLES: RESULTS OF DATING OF DETRITAL ZIRCONS FROM BASAL HORIZONS OF THE LOWER PALEOZOIC SYNRIFT COMPLEX OF THE POLAR URALS, AND IGNEOUS ROCK AGE DATA REPORT

The Upper Cambrian-Lower Ordovician terrigenous strata, unconformably overlying the Upper Riphean-Vendian rocks of the Timanide orogen in the north of the Urals, and contemporaneous bimodal volcanics and intrusive rocks are considered complexes marking the beginning of rifting which led to the opening of the Paleo-Ural Ocean later on. The article presents the results of U-Pb (LA-ICP-MS) dating of detrital zircons from sandstones lying at the base of the section of the rift complex in the Malaya Usa River basin in the Polar Urals. It was found that the clastic sequence (identified as the Khoydyshor formation) began to accumulate no earlier than the Cambrian–Ordovician transition. Zircon ages fall within the continuous Vendian to Early Ordovician (575–478 Ma) interval with peak at 512 Ma. This age range overlaps with the age of rhyolites interlayered conformably with sandstones and rhyolite porphyry dikes intruding the Khoydyshor formation, thus indicating a possible admixture of products of synsedimentary volcanism. A narrow zircon age pattern allows us to conclude that the main sources of detrital zircons in sandstones were the Early-Middle Cambrian igneous rocks formed at the stage of pre-rift uplift, and, to a lesser extent, the Late Cambrian-Early Ordovician riftogenic magmatic complexes, marking the beginning of the Uralian tectonic cycle, as well as the Late Vendian igneous rocks of the underlying Timanide orogen. Judging by the Th/U ratio, most of the detrital zircons within the sandstones were derived from the Early Paleozoic silicic volcanic and hypabyssal rocks and the Vendian granitoids and diorites. The almost complete absence of older grains, which are typical of coeval sandstones of the northern part of the Urals, may indicate the accumulation of the considered sandstones in a local trough with local clastic material sources. Probability density estimation of U-Pb ages for zircon from igneous and metamorphic rocks of the Polar Urals indicates that there were no gaps in the Late Riphean to Early Ordovician endogenous activity in this region. The main peaks occur at 552, 521 and 500 Ma, and an additional peak – at 665 Ma. The results of dating of detrital zircons from sandstones of the Khoydyshor formation together with the database of U-Pb isotope ages of igneous and metamorphic rocks of the Polar Urals (119 items), compiled by the authors, indicate that the change in geodynamic regime from collisional orogenesis in the Late Vendian to the Early-Middle Cambrian pre-rift uplift and following Late Cambrian rifting was not accompanied by a longterm discontinuity in magmatic activity.

Failure precursors recognition method for loading coal and rock using the fracture texture features of infrared thermal images

Early warning of the catastrophic failure of rocks is a challenging rock mechanics problem. This article proposed a failure precursor recognition method based on infrared thermal image texture features for coal and rock. Based on spatiotemporal background noise correction for infrared thermal images, a new thermal image parameter of loading coal and rock, Contrast Texture Feature Values (CTFV), is proposed to extract subtle changes in the thermal image caused by crack evolution based on the Gray Level Co-occurrence Matrix (GLCM). The cumulative Crack Texture Thermal Image (CTTI) is reconstructed using CTFV, which can accurately reflect the spatial evolution process of loading coal and rock cracks. The CTFV remains at 0 in the early stage of loading and gradually increases with stress increase at the unstable crack propagation stage, which can serve as a reference for precursor warning of coal and rock failure. For shale, sandstone, and limestone, the precursor of CTFV at grayscale levels of 7, 8, and 9, can be classified into high failure risk, medium failure risk, and low failure risk, respectively. For coal samples, the CTFV is only applicable for the critical warning of high failure risk when the grayscale level is 7. Then, an adaptive identification method for failure precursors based on the sliding window probability density estimation method is proposed. The research results can enhance the reliability of IR monitoring technology for rock failure and instability early warning and can provide support for the prevention and control of rock engineering and geological disasters.

Inventory, Dynamic Evolution, and Scenario Projections of Agricultural Carbon Emissions in Shandong Province, China

The reduction in agricultural carbon emissions (ACEs) in Shandong Province is essential to China’s carbon peak and carbon neutrality objectives. In this regard, we constructed an ACE inventory for Shandong Province at a resolution of 1 km × 1 km, integrating the emission factor method with geographic information system (GIS) technology. Building upon this, we explored the dynamic evolution patterns of ACEs using kernel density estimation and conditional probability density estimation. Additionally, long short-term memory networks were trained to predict ACEs under various scenarios. The results showed that: (1) ACEs in Shandong Province exhibited two stages of change, i.e., “rise and decline”. Notably, 64.39% of emissions originated from the planting industry. The distribution of emissions was closely correlated with regional agricultural production modes. Specifically, CO2 emissions were predominantly distributed in crop cultivation areas, while CH4 and N2O emissions were primarily distributed in livestock breeding areas. The uncertainty of the emission inventory ranged from −12.04% to 10.74%, mainly caused by emission factors. (2) The ACE intensity of various cities in Shandong Province is decreasing, indicating a decoupling between ACEs and agricultural economic growth. Furthermore, the emission disparities among different cities are diminishing, although significant spatial non-equilibrium still persists. (3) From 2022 to 2030, the ACEs in Shandong Province will show a continuous downward trend. By 2030, the projected values under the baseline scenario, low-carbon scenario I, and low-carbon scenario II will be 6301.74 × 104 tons, 5980.67 × 104 tons, and 5850.56 × 104 tons. The low-carbon scenario reveals greater potential for ACE reduction while achieving efficient rural economic development and urbanization simultaneously. This study not only advances the methodology of the ACE inventory but also provides quantitative references and scientific bases for promoting low-carbon, efficient, and sustainable regional agriculture.

Particle filtering supported probability density estimation of mobility patterns

This paper presents a methodology that aims to enhance the accuracy of probability density estimation in mobility pattern analysis by integrating prior knowledge of system dynamics and contextual information into the particle filter algorithm. The quality of the data used for density estimation is often inadequate due to measurement noise, which significantly influences the distribution of the measurement data. Thus, it is crucial to augment the information content of the input data by incorporating additional sources of information beyond the measured position data. These other sources can include the dynamic model of movement and the spatial model of the environment, which influences motion patterns. To effectively combine the information provided by positional measurements with system and environment models, the particle filter algorithm is employed, which generates discrete probability distributions. By subjecting these discrete distributions to exploratory techniques, it becomes possible to extract more certain information compared to using raw measurement data alone. Consequently, this study proposes a methodology in which probability density estimation is not solely based on raw positional data but rather on probability-weighted samples generated through the particle filter. This approach yields more compact and precise modeling distributions. Specifically, the method is applied to process position measurement data using a nonparametric density estimator known as kernel density estimation. The proposed methodology is thoroughly tested and validated using information-theoretic and probability metrics. The applicability of the methodology is demonstrated through a practical example of mobility pattern analysis based on forklift data in a warehouse environment.

Weighted Probability Density Estimator with Updated Bandwidths

Weighted Probability Density Estimator with Updated Bandwidths

Two-Phase Adaptive Kriging Model Based Importance Sampling Method for Estimating Time-Dependent Failure Probability

To improve the computational efficiency of the time-dependent failure probability (TDFP) to system with dynamic loads, materials degeneration and other time-varying factors, a two-phase adaptive Kriging model-based importance sampling (IS) method is proposed. The main contribution of this article is that a single Kriging model is established and adaptively refined to gradually approach the optimal IS probability density function (PDF) and estimate TDFP in two sequential phases. In the first phase, a generally approximate IS PDF is constructed by Kriging model based on its prediction characteristic and subsequently updated until convergence. Then, a simple rejection sampling technique is used to robustly generate IS samples for the estimation of TDFP. In the second phase, current Kriging model is subsequently refined for accurately identifying the classification of the IS samples generated in the last phase. Finally, the TDFP can be estimated using the products obtained in the above two phases. As the IS PDF established by Kriging model is used for directly approximating the optimal one, the proposed method has the capacity to deal with the complicated problem containing multiple failure regions. Additionally, since IS procedure can dramatically improve the sampling efficiency, the proposed method is an effective technique to the time-consuming TDFP analysis problems.

Probability Density Estimation for Composite Systems Reliability Indices Using Cross-Entropy Based Continuous Time Markov Chain Simulation

Probability Density Estimation for Composite Systems Reliability Indices Using Cross-Entropy Based Continuous Time Markov Chain Simulation

Anomaly detection in the presence of irrelevant features

Experiments at particle colliders are the primary source of insight into physics at microscopic scales. Searches at these facilities often rely on optimization of analyses targeting specific models of new physics. Increasingly, however, data-driven model-agnostic approaches based on machine learning are also being explored. A major challenge is that such methods can be highly sensitive to the presence of many irrelevant features in the data. This paper presents Boosted Decision Tree (BDT)-based techniques to improve anomaly detection in the presence of many irrelevant features. First, a BDT classifier is shown to be more robust than neural networks for the Classification Without Labels approach to finding resonant excesses assuming independence of resonant and non-resonant observables. Next, a tree-based probability density estimator using copula transformations demonstrates significant stability and improved performance over normalizing flows as irrelevant features are added. The results make a compelling case for further development of tree-based algorithms for more robust resonant anomaly detection in high energy physics.

Maximum likelihood estimation of log-concave densities on tree space

Phylogenetic trees are key data objects in biology, and the method of phylogenetic reconstruction has been highly developed. The space of phylogenetic trees is a nonpositively curved metric space. Recently, statistical methods to analyze samples of trees on this space are being developed utilizing this property. Meanwhile, in Euclidean space, the log-concave maximum likelihood method has emerged as a new nonparametric method for probability density estimation. In this paper, we derive a sufficient condition for the existence and uniqueness of the log-concave maximum likelihood estimator on tree space. We also propose an estimation algorithm for one and two dimensions. Since various factors affect the inferred trees, it is difficult to specify the distribution of a sample of trees. The class of log-concave densities is nonparametric, and yet the estimation can be conducted by the maximum likelihood method without selecting hyperparameters. We compare the estimation performance with a previously developed kernel density estimator numerically. In our examples where the true density is log-concave, we demonstrate that our estimator has a smaller integrated squared error when the sample size is large. We also conduct numerical experiments of clustering using the Expectation-Maximization algorithm and compare the results with k-means++ clustering using Fréchet mean.

Cross-entropy based importance sampling for composite systems reliability evaluation with consideration of multivariate dependence

Multivariate kernel density estimation (KDE) is powerful for dependence modeling, but producing a non-sparse probability density estimation model. To address this issue, a sparse density estimation technique termed as generalized cross-entropy (GCE) density estimator, is proposed to estimate the joint probability density function (PDF) of correlated random variables (RVs). Moreover, to effectively accelerate the composite systems reliability evaluation, a unified cross-entropy based importance sampling (CEIS) method accommodating both GCE and KDE density estimators is presented, and a creative CE optimization approach is also developed based on the composition sampling algorithm, through which the analytical updating rules of IS-PDF parameters can be derived even though GCE and KDE estimators do not belong to the exponential distribution family (EF). Compared to the KDE estimator, the GCE estimator encounters much less dimensionality difficulty in CE optimization due to the model sparsity merit, and consequently achieves more accurate IS-PDF parameters and better CEIS efficiency. The superiority of GCE estimator over KDE estimator in correlation modeling and CEIS is verified by several numerical tests.

Analysis of optimization methods for nonparametric estimation of probability density in large samples

Analysis of optimization methods for nonparametric estimation of probability density in large samples

Estimation of potential wildfire behavior characteristics to assess wildfire danger in southwest China using deep learning schemes

Accurate estimation of potential wildfire behavior characteristics (PWBC) can improve wildfire danger assessment. However, wildfire behavior has been estimated by most fire spread models with immeasurable uncertainties and difficulties in large-scale applications. In this study, a PWBC estimation model (named PWBC-QR-BiLSTM) was proposed by coupling the Bi-directional Long Short-Term Memory (BiLSTM) and quantile regression (QR) methods. Multi-source data, including fuel, weather, topography, infrastructure, and landscape variables, were input into the PWBC-QR-BiLSTM model to estimate the potential rate of spread (ROS) and fire radiative power (FRP) over western Sichuan of China, and then to estimate the probability density of ROS and FRP. Daily ROS and FRP were extracted from the Global Fire Atlas and the MOD14A1/MYD14A1 product. The optimal PWBC-QR-BiLSTM model was determined using the Non-dominated Sorting Genetic Algorithm II (NSGA-II). Results showed that the PWBC-QR-BiLSTM performed well in estimating potential ROS and FRP with high accuracy (ROS: R2 > 0.7 and MAPE<30%, FRP: R2 > 0.8 and MAPE<25%). The modal PWBC values extracted from the estimated probability density were closer to the observed values, which can be regarded as a good indicator for wildfire danger assessment. The variable importance analysis also verified that fuel and infrastructure variables played an important role in driving wildfire behavior. This study suggests the potential of utilizing artificial intelligence to estimate PWBC and its probability density to improve the guidance on wildfire management.

Wavelet Probabilistic Neural Networks.

In this article, a novel wavelet probabilistic neural network (WPNN), which is a generative-learning wavelet neural network that relies on the wavelet-based estimation of class probability densities, is proposed. In this new neural network approach, the number of basis functions employed is independent of the number of data inputs, and in that sense, it overcomes the well-known drawback of traditional probabilistic neural networks (PNNs). Since the parameters of the proposed network are updated at a low and constant computational cost, it is particularly aimed at data stream classification and anomaly detection in off-line settings and online environments where the length of data is assumed to be unconstrained. Both synthetic and real-world datasets are used to assess the proposed WPNN. Significant performance enhancements are attained compared to state-of-the-art algorithms.

RSS-Bagging: Improving Generalization Through the Fisher Information of Training Data.

The bagging method has received much application and attention in recent years due to its good performance and simple framework. It has facilitated the advanced random forest method and accuracy-diversity ensemble theory. Bagging is an ensemble method based on simple random sampling (SRS) method with replacement. However, SRS is the most foundation sampling method in the field of statistics, where exists some other advanced sampling methods for probability density estimation. In imbalanced ensemble learning, down-sampling, over-sampling, and SMOTE methods have been proposed for generating base training set. However, these methods aim at changing the underlying distribution of data rather than simulating it better. The ranked set sampling (RSS) method uses auxiliary information to get more effective samples. The purpose of this article is to propose a bagging ensemble method based on RSS, which uses the ordering of objects related to the class to obtain more effective training sets. To explain its performance, we give a generalization bound of ensemble from the perspective of posterior probability estimation and Fisher information. On the basis of RSS sample having a higher Fisher information than SRS sample, the presented bound theoretically explains the better performance of RSS-Bagging. The experiments on 12 benchmark datasets demonstrate that RSS-Bagging statistically performs better than SRS-Bagging when the base classifiers are multinomial logistic regression (MLR) and support vector machine (SVM).

Sensitivity-Aware Density Estimation in Multiple Dimensions.

We formulate an optimization problem to estimate probability densities in the context of multidimensional problems that are sampled with uneven probability. It considers detector sensitivity as an heterogeneous density and takes advantage of the computational speed and flexible boundary conditions offered by splines on a grid. We choose to regularize the Hessian of the spline via the nuclear norm to promote sparsity. As a result, the method is spatially adaptive and stable against the choice of the regularization parameter, which plays the role of the bandwidth. We test our computational pipeline on standard densities and provide software.We also present a new approach to PET rebinning as an application of our framework.

A novel probabilistic forecasting system based on quantile combination in electricity price

In the electricity market, the accuracy of electricity price forecasting is significant for real-time control; however, the complexity and volatility of electricity prices make this a challenge. Existing forecasting models focus on deterministic forecasting and rarely address the uncertainty in electricity price forecasting. Therefore, this study fills this knowledge gap by introducing a novel combined probability forecasting system (CPFS) and creatively incorporating probability density estimation based on kernel functions in a multi-objective optimization algorithm. In addition, to effectively integrate the forecasting components, the tuna optimization algorithm was enhanced to overcome the limitations of traditional multi-objective optimization algorithms. Finally, the validity of the CPFS is confirmed through two electricity price cases, considering three equally important aspects: reliability, resolution, and sharpness. From a comprehensive perspective, CPFS outperformed the most advanced benchmark by more than 5.66% and 38.93% in AIS and by more than 13.41% and 3.55% in quantile loss on the NSW and Singapore datasets, respectively. The experimental results demonstrate that the CPFS provides an effective range for electricity price fluctuations. Furthermore, given that probabilistic forecasting is essential for risk management, it offers important implications for the electricity market.