Probability Distribution Of Samples Research Articles

Robust estimation of smooth graph signals from randomized space–time samples

Abstract Heat diffusion processes have found wide applications in modelling dynamical systems over graphs. In this paper, we consider the recovery of a $k$-bandlimited graph signal that is an initial signal of a heat diffusion process from its space–time samples. We propose three random space–time sampling regimes, termed dynamical sampling techniques, that consist in selecting a small subset of space–time nodes at random according to some probability distribution. We show that the number of space–time samples required to ensure stable recovery for each regime depends on a parameter called the spectral graph weighted coherence, which depends on the interplay between the dynamics over the graphs and sampling probability distributions. In optimal scenarios, as little as $\mathcal{O}(k \log (k))$ space–time samples are sufficient to ensure accurate and stable recovery of all $k$-bandlimited signals. Dynamical sampling typically requires much fewer spatial samples than the static case by leveraging the temporal information. Then, we propose a computationally efficient method to reconstruct $k$-bandlimited signals from their space–time samples. We prove that it yields accurate reconstructions and that it is also stable to noise. Finally, we test dynamical sampling techniques on a wide variety of graphs. The numerical results on synthetic and real climate datasets support our theoretical findings and demonstrate the efficiency.

MELCOR – DAKOTA coupling for uncertainty analyses in the SNAP environment/architecture

Uncertainty estimation to assess figures of merit characterizing evolution of a severe accident transient is a topic of current investigation in development of best-estimate plus uncertainty methodology. The probabilistic method to propagate input uncertainty is one of the methodologies used to develop Uncertainty Analyses (UAs). Using this methodology, UAs are performed by sampling probability distributions that describe the range of possible values that computer simulation model inputs can have. For each sample (or realization) of a set of uncertain input parameters, a computer simulation is performed. From the range of code simulation results obtained for each input realization, a distribution of code results is obtained. In this process, the distribution of input uncertainties is propagated to obtain a distribution of possible code results (i.e., the code output uncertainty). This probabilistic methodology is facilitated using Uncertainty Tools (UTs), which can be coupled with the accident analysis computer code to perform an UA. One of the UTs currently available is DAKOTA, developed by Sandia National Laboratories. DAKOTA is also provided as a SNAP plug-in. SNAP is a graphical user interface designed to support the use of USNRC codes (MELCOR, TRACE, etc). This paper is entirely derived from the NUREG/IA-532 issued by USNRC in April 2023 (Mascari et al., 2023) and has as a major target to summarize the main needs of UA in severe accident, the main element of the probabilistic method to propagate input uncertainty, and the workflow within SNAP to assist other interested analysts with their applications given they are members of the USNRC Cooperative Severe Accident Research Program (CSARP).

Entropy-Optimized Deep Weighted Product Quantization for Image Retrieval.

Hashing and quantization have greatly succeeded by benefiting from deep learning for large-scale image retrieval. Recently, deep product quantization methods have attracted wide attention. However, representation capability of codewords needs to be further improved. Moreover, since the number of codewords in the codebook depends on experience, representation capability of codewords is usually imbalanced, which leads to redundancy or insufficiency of codewords and reduces retrieval performance. Therefore, in this paper, we propose a novel deep product quantization method, named Entropy Optimized deep Weighted Product Quantization (EOWPQ), which not only encodes samples into the weighted codewords in a new flexible manner but also balances the codeword assignment, improving while balancing representation capability of codewords. Specifically, we encode samples using the linear weighted sum of codewords instead of a single codeword as traditionally. Meanwhile, we establish the linear relationship between the weighted codewords and semantic labels, which effectively maintains semantic information of codewords. Moreover, in order to balance the codeword assignment, that is, avoiding some codewords representing most samples or some codewords representing very few samples, we maximize the entropy of the coding probability distribution and obtain the optimal coding probability distribution of samples by utilizing optimal transport theory, which achieves the optimal assignment of codewords and balances representation capability of codewords. The experimental results on three benchmark datasets show that EOWPQ can achieve better retrieval performance and also show the improvement of representation capability of codewords and the balance of codeword assignment.

Bounded exponential loss function based AdaBoost ensemble of OCSVMs

As a commonly used ensemble method, AdaBoost has drawn much consideration in the field of machine learning. However, AdaBoost is highly sensitive to outliers. The performance of AdaBoost may be greatly deteriorated when the training samples are polluted by outliers. For binary and multi-class classifications, there have emerged many approaches to improving the robustness of AdaBoost against outliers. Unfortunately, there are too few researches on enhancing the robustness of AdaBoost against outliers in the case of one-class classification. In this study, the exponential loss function of AdaBoost is replaced by a more robust one to improve the anti-outlier ability of the conventional AdaBoost based ensemble of one-class support vector machines (OCSVMs). Furthermore, based on the redesigned loss function, the update formulae for the weights of base classifiers and the probability distribution of training samples are reformulated towards the AdaBoost ensemble of OCSVMs. The empirical error upper bound is derived from the theoretical viewpoint. Experimental outcomes upon the artificial and benchmark data sets show that the presented ensemble approach is more robust against outliers than its related methods.

Multi-Angle Reliability Evaluation of Grid-Connected Wind Farms with Energy Storage Based on Latin Hypercube Important Sampling

Aiming to combat the problems of slow speed and poor accuracy of reliability evaluation of the power system in wind farms with energy storage, this paper proposes a method of reliability evaluation based on Latin hypercube important sampling (LHIS). Firstly, we aimed to establish the Latin hypercube important sampling evaluation model by combining the Latin hypercube sampling method with the important sampling method. Secondly, we aimed to optimize the sample probability distribution of the components and conduct hierarchical sampling of the system. Then, the comprehensive risk indicator (CRI) was proposed to evaluate the operational risk and the wind storage generation interrupted energy benefit (WSGIEB) was proposed to evaluate the contribution of the reliability. Finally, simulation experiments were carried out through various power system operation scenarios. The simulation results show that the proposed method is 47% higher than the improving importance sampling method (IM-IS) in evaluation speed and 33% higher than the improving importance sampling method in calculation accuracy.

Posterior Assessment of Parameters in a Time Domain Random Walk Model of Partitioning Tracer Tests in Two‐Phase Flow Scenarios

AbstractWe provide a reliable and efficient methodological framework for the interpretation of laboratory‐scale partitioning tracer test data under uncertainty. The proposed approach rests on a Time domain random walk (TDRW) particle tracking methodology. The range of applicability of the latter is extended to include transport of partitioning tracers upon considering retardation and trapping mechanisms. A classical maximum likelihood (ML) approach is applied considering the extensive set of experimental observations of Dwarakanath et al. (1999, https://doi.org/10.1021/es990082v). This yields best estimates of model parameters, including residual immobile phase saturation, the partition coefficient and the parameters of the memory function employed to simulate the impact of solute trapping. Experimental observations of the partition coefficient are included in the objective function upon relying on a regularization term. We show that considering these types of information, which are typically obtained through batch experiments, is important to attain joint estimates of the partition coefficient and of residual immobile phase saturation. Sample probability distributions of model parameters conditional on available data are then assessed through a stochastic inverse modeling approach. This step poses a significant challenge in terms of computational effort and is performed through a reduced order surrogate model. Our results show that the TDRW‐based approach can effectively capture the key features of the observed breakthrough curves of the various partitioning tracers analyzed and provide satisfactory estimates of residual immobile phase saturation.

Generative and discriminative training of Boltzmann machine through quantum annealing

A hybrid quantum-classical method for learning Boltzmann machines (BM) for a generative and discriminative task is presented. BM are undirected graphs with a network of visible and hidden nodes where the former is used as the reading site. In contrast, the latter is used to manipulate visible states’ probability. In Generative BM, the samples of visible data imitate the probability distribution of a given data set. In contrast, the visible sites of discriminative BM are treated as Input/Output (I/O) reading sites where the conditional probability of output state is optimized for a given set of input states. The cost function for learning BM is defined as a weighted sum of Kullback-Leibler (KL) divergence and Negative conditional Log-likelihood (NCLL), adjusted using a hyper-parameter. Here, the KL Divergence is the cost for generative learning, and NCLL is the cost for discriminative learning. A Stochastic Newton-Raphson optimization scheme is presented. The gradients and the Hessians are approximated using direct samples of BM obtained through quantum annealing. Quantum annealers are hardware representing the physics of the Ising model that operates on low but finite temperatures. This temperature affects the probability distribution of the BM; however, its value is unknown. Previous efforts have focused on estimating this unknown temperature through regression of theoretical Boltzmann energies of sampled states with the probability of states sampled by the actual hardware. These approaches assume that the control parameter change does not affect the system temperature; however, this is usually untrue. Instead of using energies, the probability distribution of samples is employed to estimate the optimal parameter set, ensuring that the optimal set can be obtained from a single set of samples. The KL divergence and NCLL are optimized for the system temperature, and the result is used to rescale the control parameter set. The performance of this approach, as tested against the theoretically expected distributions, shows promising results for Boltzmann training on quantum annealers.

On Iterated Interpolation

Matrices resulting from the discretization of a kernel function, e.g., in the context of integral equations or sampling probability distributions, can frequently be approximated by interpolation. In order to improve the efficiency, a multi-level approach can be employed that involves interpolating the kernel functions and its approximations multiple times. This article presents a new approach to analyze the error incurred by these iterated interpolation procedures that is considerably more elegant than its predecessors and allows us to treat not only the kernel function itself, but also its derivatives.

Chaotic Image Encryption Based on Boson Sampling

AbstractAs a fundamental form of information representation, image encryption is a significant research issue in the multimedia era. An image encryption scheme is proposed benefiting from chaotic random behavior characteristics of Boson sampling (BS) probability distribution, where the chaotic random number sequence is used to implement the encryption process including scrambling image pixels and altering the values of pixels. Further, a BS‐based image encryption prototype system is constructed on the programmable silicon photonic processor chip for BS with 5‐mode 2‐photon, which can encrypt 4 × 4 grayscale images. Besides, experiments for encrypting 256 × 256 gray‐scale images with 20‐mode 8‐photon BS are simulated on the StrawberryFields photonic quantum software platform. Numerical analyses demonstrate the proposed scheme has good effectiveness in terms of randomness test, information entropy, histogram, correlation, key space, key sensitivity, and anti‐attack, offering gains in many cryptographic applications. The BS‐based image encryption prototype system extends the application of BS to solve practical problems in addition to proving quantum advantages.

Data-Correlation-Aware Unsupervised Deep-Learning Model for Anomaly Detection in Cyber–Physical Systems

A cyber–physical system (CPS) is a multidimensional complex system integrating computing, communication, and control technologies. Because of their key functionality within the system, CPS requires large robustness and security to ensure its reliable operation. Due to its importance in supporting overall system security, anomaly detection (AD) is likely to continue to play an important role in the CPS security. Moreover, unsupervised AD models based on deep learning have shown better performances in rule training, adaptive update, detection efficiency, and accuracy. Due to the nature of the CPS systems, CPS data is more likely to exhibit implicit correlative relationship among data, which would be vital to exploit for CPS security provisions in more complex data environments. In view of this observation, we propose the data-correlation-aware unsupervised deep learning model for AD in CPS, which uses an undigraph structure to store samples and implicit correlation among samples. We design a dual-autoencoder to train both original features and implicit correlation features among data, and we construct an estimation network using Gaussian mixture model (GMM) to evaluate the probability distribution of samples to complete the anomaly analysis. Experimental results compared the performance with relevant AD models based on deep learning which did not use data-correlation analysis. The results showed that, under some representative application scenarios of CPS considered, data-correlation-aware unsupervised deep-learning model achieved superior results in parameter sensitivity, ablation, relationship between correlation degree and detection performance, visualization, and detection effects.

FABNet: Fusion Attention Block and Transfer Learning for Laryngeal Cancer Tumor Grading in P63 IHC Histopathology Images.

Laryngeal cancer tumor (LCT) grading is a challenging task in P63 Immunohistochemical (IHC) histopathology images due to small differences between LCT levels in pathology images, the lack of precision in lesion regions of interest (LROIs) and the paucity of LCT pathology image samples. The key to solving the LCT grading problem is to transfer knowledge from other images and to identify more accurate LROIs, but the following problems occur: 1) transferring knowledge without a priori experience often causes negative transfer and creates a heavy workload due to the abundance of image types, and 2) convolutional neural networks (CNNs) constructing deep models by stacking cannot sufficiently identify LROIs, often deviate significantly from the LROIs focused on by experienced pathologists, and are prone to providing misleading second opinions. So we propose a novel fusion attention block network (FABNet) to address these problems. First, we propose a model transfer method based on clinical a priori experience and sample analysis (CPESA) that analyzes the transfer ability by integrating clinical a priori experience using indicators such as the relationship between the cancer onset location and morphology and the texture and staining degree of cell nuclei in histopathology images; our method further validates these indicators by the probability distribution of cancer image samples. Then, we propose a fusion attention block (FAB) structure, which can both provide an advanced non-uniform sparse representation of images and extract spatial relationship information between nuclei; consequently, the LROI can be more accurate and more relevant to pathologists. We conducted extensive experiments, compared with the best Baseline model, the classification accuracy is improved 25%, and It is demonstrated that FABNet performs better on different cancer pathology image datasets and outperforms other state of the art (SOTA) models.

Multiple projection Markov chain Monte Carlo algorithms on submanifolds

Abstract We propose new Markov chain Monte Carlo (MCMC) algorithms to sample probability distributions on submanifolds, which generalize previous methods by allowing the use of set-valued maps in the proposal step of the MCMC algorithms. The motivation for this generalization is that the numerical solvers used to project proposed moves to the submanifold of interest may find several solutions. We show that the new algorithms indeed sample the target probability measure correctly, thanks to some carefully enforced reversibility property. We demonstrate the interest of the new MCMC algorithms on illustrative numerical examples.

Predicting the peak particle velocity from rock blasting operations using Bayesian approach

Measuring the blast-induced ground vibration at blasting sites is very important, to plan and avoid adverse effects of blasting in terms of the peak particle velocity (PPV). However, the measurement of PPV often requires time, cost, and logistic commitment, which may not be economical for small-scale mining operations. This has prompted the development of numerous regression equations in the literature to estimate PPV from a relatively easier to estimate scaled distance (SD) measurement. With numerous regression equations available in the literature, there is a challenge of how to select the appropriate model for a specific blasting site, more so that rocks behave differently from site to site because of different geological processes that rocks are subjected to. This study develops a method that selects appropriate models for specific blasting sites by comparing the evidence and occurrence probability of different regression models. The appropriate model is the model with the highest evidence and occurrence probability given the available blasting site SD data. The selected model is then integrated with prior knowledge and available blasting SD data in Bayesian framework for probabilistic characterization of PPV. The SD and PPV data at the opencast coal mine, Jharia coalfield in the Dhanbad district of Jharkhand, India, is used to illustrate and validate the approach. The mean and standard deviation of simulated PPV samples from the proposed approach are 12.38 mm/s and 7.36 mm/s, respectively, which are close to the mean of 12.03 mm/s and standard deviation of 9.24 mm/s estimated from the measured PPV at the site. In addition, the probability distribution of the simulated PPV samples is consistent with the probability distribution of the measured PPV at the blasting site.

Dynamic analysis of an L-shaped liquid-filled pipe with interval uncertainty

In this paper, a fourteen-equation fluid-structure coupling model of an L-shaped liquid-filled pipe with elastic support is established by the transfer matrix method (TMM). The validity of the dynamic model of the liquid-filled pipe is verified by comparing the frequency results obtained from the literature, experiment, and finite element method. The liquid-filled pipe system is inevitably subject to uncertainties due to the manufacture and assembly error, material dispersion, and the change of working conditions. To obtain more reasonable evaluations of the dynamic response of the L-shaped liquid-filled pipe, an uncertain frequency response surrogate function (UFRSF) is proposed to evaluate the uncertainty of the liquid-filled pipe. The pipe parameters, support parameters, and fluid parameters are taken as unknown but bounded interval variables. The frequency responses of the L-shaped liquid-filled pipe under various uncertain variables are calculated. Compared with the Monte Carlo simulation (MCS) results, the efficiency and accuracy of the proposed method for solving the uncertain frequency response of liquid-filled pipe are proved. The proposed method has the advantages of convenience in implementation, high efficiency and transparency, and can provide theoretical guidance for the uncertainty assessment of liquid-filled pipes with unknown sample probability distribution in engineering.

Stochastic generation of runoff series for multiple reservoirs based on generative adversarial networks

The implicit stochastic optimization (ISO) model is a methodology that has been extensively employed in the mid-long term optimal operation of a reservoir system. The generation of runoff series is the key technology to make up for the poor representativeness and reliability of historical runoff samples and is the premise to guarantee the accuracy of the ISO model. However, the traditional methods are not suitable for the stochastic generation of runoff series, among reservoirs with spatial–temporal correlation. In this work, the generative adversarial network called DC-WGAN is developed as a runoff generation model. DC-WGAN is based on a data-driven model and can capture the correlation of runoff series in time and space dimensions simultaneously. The proposed method is applied to the downstream Jinsha river cascade and Three Gorges cascade reservoirs in China to generate ten-day runoff series and is compared with the stochastic generation method based on Copula function. The focus of this paper is the model's performance in replicating the spatial–temporal correlation of ten-day runoff series. Results are as follows. (1) DC-WGAN has strong learning and generalization ability, which can learn the correct distribution of original runoff samples and generate runoff series similar to that from the validation set. (2) The runoff series among reservoirs with spatial–temporal correlation can be generated through DC-WGAN without assuming the probability distribution of samples in advance. (3) DC-WGAN is better than Copula function in capturing the spatial–temporal correlation, especially the moderate and weak correlation. The proposed methodology has a high practical value and application prospect for the stochastic generation of runoff series with spatial–temporal correlation.

Binary imbalanced data classification based on diversity oversampling by generative models

In many practical applications, the data are class imbalanced. Accordingly, it is very meaningful and valuable to investigate the classification of imbalanced data. In the framework of binary imbalanced data classification, the synthetic minority oversampling technique (SMOTE) is the best-known oversampling method. However, for each positive sample, SMOTE generates only k synthetic samples on the lines between the positive sample and its k-nearest neighbors, resulting in three drawbacks: (1) SMOTE cannot effectively extend the training field of positive samples; (2) the generated positive samples lack diversity; (3) SMOTE does not accurately approximate the probability distribution of the positive samples. Therefore, two binary imbalanced data classification methods named BIDC1 and BIDC2 based on diversity oversampling by generative models are proposed. The BIDC1 and BIDC2 conduct diversity oversampling using extreme learning machine autoencoder and generative adversarial network, respectively. Extensive experiments on 26 data sets are conducted to compare the two methods with 14 state-of-the-art methods using five metrics: F-measure, G-means, AUC-area, MMD-score, and Silhouette-score. The experimental results demonstrate that the two methods outperform the other 14 methods.

A new multivariate equation-error autoregressive moving average system with conditional heteroscedastic noise: Maximum likelihood identification

The aim of this paper is to deal with the problem of conditional heteroscedastic noise in multivariate systems. In this regard, a new multivariate equation-error system with colored noise is introduced, in which the noise conditional variance varies with time and the noise exhibits a GARCH process. Thus, in this our new approach, both the time dependency and probability distribution of noise samples shall be identified. Based on the maximum likelihood principle and orthogonality of parameters, the problem of multivariate system identification is reduced into two separate maximum likelihood problems. An iterative algorithm is proposed, which can efficiently estimate the problem parameters based on gradient reduction and nonlinear optimization. Besides, a bootstrap algorithm is applied for assessing the accuracy of estimators. Also, some asymptotic result of proposed method in large sample size is provided. According to bootstrap simulation results, the proposed algorithm can effectively estimate the parameters of the system with conditional heteroscedastic noise and provides more accurate parameter estimates with a much lower confidence interval, compared to the least square method.

Similarity Network Fusion Based on Random Walk and Relative Entropy for Cancer Subtype Prediction of Multigenomic Data

It is a crucial task to design an integrated method to discover cancer subtypes and understand the heterogeneity of cancer based on multiple genomic data. In recent years, some clustering algorithms have been proposed and applied to cancer subtype prediction. Among them, similarity network fusion (SNF) can integrate multiple types of genomic data to identify cancer subtypes, which improves the understanding of tumorigenesis. SNF uses a dense similarity matrix to obtain the global information of the data, and the interconnection of samples between different categories will cause noise interference. Therefore, how to construct a more robust dense similarity matrix is an important research content to improve the performance of cancer subtype identification. In this paper, we proposed similarity network fusion based on random walk and relative entropy (R2SNF) for cancer subtype prediction. Firstly, the random walk algorithm was used to capture the complex relationship between samples in each genomic data. And the transition probability distribution of samples in the network was obtained. If two samples belong to the same class, the transition probability between the two samples is great. On the contrary, if the two samples do not belong to the same class, the transition probability between the two samples is small. In this way, the degree of correlation between samples can be well obtained, thereby reducing the noise interference caused by the interconnection of samples between different categories. Secondly, relative entropy was used to calculate the difference in the transition probability distribution between samples to construct a better dense similarity matrix which contains structural similarity information between samples. Thirdly, we iteratively fused the obtained dense similarity matrix with the KNN similarity matrix to construct the fused similarity matrix of all genomic data. Finally, by using spectral clustering, the fused similarity matrix was grouped into multiple clusters, which indicates the cancer subtypes. Experiments on seven cancer omics datasets show that the R2SNF algorithm performs well in identifying cancer subtypes.

Temporal fluctuations in chemotaxis gain implement a simulated-tempering strategy for efficient navigation in complex environments

SummaryBacterial chemotaxis is a major testing ground for systems biology, including the role of fluctuations and individual variation. Individual bacteria vary in their tumbling frequency and adaptation time. Recently, large cell-cell variation was also discovered in chemotaxis gain, which determines the sensitivity of the tumbling rate to attractant gradients. Variation in gain is puzzling, because low gain impairs chemotactic velocity. Here, we provide a functional explanation for gain variation by establishing a formal analogy between chemotaxis and algorithms for sampling probability distributions. We show that temporal fluctuations in gain implement simulated tempering, which allows sampling of attractant distributions with many local peaks. Periods of high gain allow bacteria to detect and climb gradients quickly, and periods of low gain allow them to move to new peaks. Gain fluctuations thus allow bacteria to thrive in complex environments, and more generally they may play an important functional role for organism navigation.

Probabilistic Tractable Models in Mixed Discrete-Continuous Domains

Abstract We study the problem of the unsupervised learning of graphical models in mixed discrete-continuous domains. The problem of unsupervised learning of such models in discrete domains alone is notoriously challenging, compounded by the fact that inference is computationally demanding. The situation is generally believed to be significantly worse in discrete-continuous domains: estimating the unknown probability distribution of given samples is often limited in practice to a handful of parametric forms, and in addition to that, computing conditional queries need to carefully handle low-probability regions in safety-critical applications. In recent years, the regime of tractable learning has emerged, which attempts to learn a graphical model that permits efficient inference. Most of the results in this regime are based on arithmetic circuits, for which inference is linear in the size of the obtained circuit. In this work, we show how, with minimal modifications, such regimes can be generalized by leveraging efficient density estimation schemes based on piecewise polynomial approximations. Our framework is realized on a recent computational abstraction that permits efficient inference for a range of queries in the underlying language. Our empirical results show that our approach is effective, and allows a study of the trade-off between the granularity of the learned model and its predictive power.