Marginal Probability Distributions Research Articles

Multi-Source and Multi-modal Deep Network Embedding for Cross-Network Node Classification

In recent years, to address the issue of networked data sparsity in node classification tasks, cross-network node classification (CNNC) leverages the richer information from a source network to enhance the performance of node classification in the target network, which typically has sparser information. However, in real-world applications, labeled nodes may be collected from multiple sources with multiple modalities (e.g., text, vision, and video). Naive application of single-source and single-modal CNNC methods may result in sub-optimal solutions. To this end, in this article, we propose a model called Multi-source and Multi-modal Cross-network Deep Network Embedding (M 2 CDNE) for cross-network node classification. In M 2 CDNE, we propose a deep multi-modal network embedding approach that combines the extracted deep multi-modal features to make the node vector representations network invariant. In addition, we apply dynamic adversarial adaptation to assess the significance of marginal and conditional probability distributions between each source and target network to make node vector representations label discriminative. Furthermore, we devise to classify nodes in the target network through the related source classifier and aggregate different predictions utilizing respective network weights, corresponding to the discrepancy between each source and target network. Extensive experiments performed on real-world datasets demonstrate that the proposed M 2 CDNE significantly outperforms the state-of-the-art approaches.

A dual‐channel transferable RUL prediction method integrated with Bayesian deep learning and domain adaptation for rolling bearings

AbstractMany deep learning methods typically assume that the marginal probability distribution between the training and testing bearing data is similar or the same. However, the probability distribution of rolling bearings may deviate significantly under diverse working conditions. To address the above limitations, a novel transferable remaining useful life (RUL) prediction method integrated with Bayesian deep learning and unsupervised domain adaptation (DA) is proposed. First, the signal alignment is executed on the data after the first prediction time to maintain the same granularity and scale across both source and target domains. Second, the multi‐domain features are extracted and sent into the dual‐channel Transformer network (DCTN) incorporating the convolutional block attention module (CBAM) to adequately exploit the abundant degradation information. Then, the DA module is incorporated into the model to mitigate the distribution discrepancies of the extracted high‐level merged features between the source and target domains. Finally, by applying the variational inference method, the DCTN‐CBAM is extended to the Bayesian deep neural network, and the RUL prediction and its corresponding confidence intervals can be conveniently derived. In addition, the generalization capability and effectiveness are validated through six bidirectional transfer RUL prediction tasks across two rolling bearing datasets. The experimental results demonstrate that it could provide a more reliable RUL prediction and efficiently account for the prediction uncertainty.

A New Type of Bivariate-Gamma Probability Function

This paper develops a New Bivariate-Gamma Distribution (NBGD) and explores its mathematical and statistical properties such as marginal probability distributions, moments, product moment, covariance and correlation. The study further investigates the various special cases of the NBGD. The new distribution is more robust with additional parameters and flexible for modelling real datasets.

Three-stage transfer learning for motor imagery EEG recognition.

Motor imagery (MI) paradigms have been widely used in neural rehabilitation and drowsiness state assessment. The progress in brain-computer interface (BCI) technology has emphasized the importance of accurately and efficiently detecting motor imagery intentions from electroencephalogram (EEG). Despite the recent breakthroughs made in developing EEG-based algorithms for decoding MI, the accuracy and efficiency of these models remain limited by technical challenges posed by cross-subject heterogeneity in EEG data processing and the scarcity of EEG data for training. Inspired by the optimal transport theory, this study aims to develop a novel three-stage transfer learning (TSTL) method, which uses the existing labeled data from a source domain to improve classification performance on an unlabeled target domain. Notably, the proposed method comprises three components, namely, the Riemannian tangent space mapping (RTSM), source domain transformer (SDT), and optimal subspace mapping (OSM). The RTSM maps a symmetric positive definite matrix from the Riemannian space to the tangent space to minimize the marginal probability distribution drift. The SDT transforms the source domain to a target domain by finding the optimal transport mapping matrix to reduce the joint probability distribution differences. The OSM finally maps the transformed source domain and original target domain to the same subspace to further mitigate the distribution discrepancy. The performance of the proposed method was validated on two public BCI datasets, and the average accuracy of the algorithm on two datasets was 72.24% and 69.29%. Our results demonstrated the improved performance of EEG-based MI detection in comparison with state-of-the-art algorithms.

Unsupervised Joint Domain Adaptation for Decoding Brain Cognitive States From tfMRI Images.

Recent advances in large model and neuroscience have enabled exploration of the mechanism of brain activity by using neuroimaging data. Brain decoding is one of the most promising researches to further understand the human cognitive function. However, current methods excessively depends on high-quality labeled data, which brings enormous expense of collection and annotation of neural images by experts. Besides, the performance of cross-individual decoding suffers from inconsistency in data distribution caused by individual variation and different collection equipments. To address mentioned above issues, a Join Domain Adapative Decoding (JDAD) framework is proposed for unsupervised decoding specific brain cognitive state related to behavioral task. Based on the volumetric feature extraction from task-based functional Magnetic Resonance Imaging (tfMRI) data, a novel objective loss function is designed by the combination of joint distribution regularizer, which aims to restrict the distance of both the conditional and marginal probability distribution of labeled and unlabeled samples. Experimental results on the public Human Connectome Project (HCP) S1200 dataset show that JDAD achieves superior performance than other prevalent methods, especially for fine-grained task with 11.5%-21.6% improvements of decoding accuracy. The learned 3D features are visualized by Grad-CAM to build a combination with brain functional regions, which provides a novel path to learn the function of brain cortex regions related to specific cognitive task in group level.

Deep learning model-based reading of Chinese language and literature classics and situational experience

Abstract In this paper, a new media scenario experience model is constructed to integrate Chinese language and literature works through deep learning algorithms. Firstly, the model is trained as a whole, RBM is a special form of Boltzmann machine, and the energy function is defined by combining the input layer vector and the hidden layer vector to calculate the marginal probability distribution. Then the defined energy function can be used to derive the joint probability formula of state values, link this probability formula with the hidden layer between the neuron nodes, use the logistic function to define the energy of the whole Boltzmann machine, and finally, after repeated training to derive the entropy function of the RBM, to complete the construction of a new media-based scenario experience platform. The experimental results show that after a semester of teaching based on this platform, the scoring examination of new media scenario experience classical reading was conducted. The percentage of those who passed was 82%, and only 18% failed. Therefore, we should make full use of multimedia technology, innovate efficient Chinese language and literature teaching methods, turn the classroom into a stage, enhance the novelty of Chinese language and literature courses, and enhance the intuitiveness of classical reading in Chinese language and literature.

An iterative multi-fidelity scheme for simulating multi-dimensional non-Gaussian random fields

This paper presents an efficient multi-fidelity scheme to simulate multi-dimensional non-Gaussian random fields that are specified by covariance functions and marginal probability distribution functions. To develop the multi-fidelity scheme, two numerical algorithms are proposed in turn. The first algorithm is used to simulate random samples of the random field. In this algorithm, initial random samples are first generated to meet the marginal distribution and an iterative procedure is adopted to change the ranking of the random samples to match the target covariance function. By using the above algorithm, random samples satisfying the target covariance function and the target marginal distribution are obtained, but it is computationally intensive and is not suitable to simulate large-scale random fields. By taking advantage of Karhunen–Loève expansion, a multi-fidelity algorithm is then proposed to reduce the computational effort. The random variables in Karhunen–Loève expansion are calculated via performing the first algorithm on the low-fidelity model and the deterministic functions in Karhunen–Loève expansion are solved on the high-fidelity model. In this way, the proposed method has low computational effort and a high fidelity simultaneously. Three numerical examples, including two- and three-dimensional non-Gaussian random fields, are used to verify the good performance of the proposed algorithms.

Pseudo Labels for Unsupervised Domain Adaptation: A Review

Conventional machine learning relies on two presumptions: (1) the training and testing datasets follow the same independent distribution, and (2) an adequate quantity of samples is essential for achieving optimal model performance during training. Nevertheless, meeting these two assumptions can be challenging in real-world scenarios. Domain adaptation (DA) is a subfield of transfer learning that focuses on reducing the distribution difference between the source domain (Ds) and target domain (Dt) and subsequently applying the knowledge gained from the Ds task to the Dt task. The majority of current DA methods aim to achieve domain invariance by aligning the marginal probability distributions of the Ds. and Dt. Recent studies have pointed out that aligning marginal probability distributions alone is not sufficient and that alignment of conditional probability distributions is equally important for knowledge migration. Nonetheless, unsupervised DA presents a more significant difficulty in aligning the conditional probability distributions because of the unavailability of labels for the Dt. In response to this issue, there have been several proposed methods by researchers, including pseudo-labeling, which offer novel solutions to tackle the problem. In this paper, we systematically analyze various pseudo-labeling algorithms and their applications in unsupervised DA. First , we summarize the pseudo-label generation methods based on the single and multiple classifiers and actions taken to deal with the problem of imbalanced samples. Second, we investigate the application of pseudo-labeling in category feature alignment and improving feature discrimination. Finally, we point out the challenges and trends of pseudo-labeling algorithms. As far as we know, this article is the initial review of pseudo-labeling techniques for unsupervised DA.

Copula-Based Severity–Duration–Frequency (SDF) Analysis of Streamflow Drought in the Source Area of the Yellow River, China

In classical severity–duration–frequency (SDF) analysis, the dependence between drought characteristics is not effectively considered. The present study aims to propose the SDF relationships of streamflow drought in the source area of the Yellow River (SAYR) using a copula-based approach. Comparison of multiple time-varying threshold levels and the integration and elimination of drought events were considered. Selection of marginal probability distribution and copula-based joint probability distribution was properly conducted with multiple means. Copula-based joint and conditional probabilities were computed. The findings support carrying out integration and elimination processing on the preliminarily identified streamflow droughts through a run analysis with a time-varying threshold level of the 80% quantile of daily streamflow. The Gaussian copula was selected as the optimal model for constructing bivariate joint probability distribution, with generalized extreme value and log-normal as the suitable marginal probability distributions of streamflow drought duration and severity. The proposed copula-based SDF relationships of streamflow drought events can provide more critical information in addition to univariate frequency analysis, benefitting from the joint and conditional probabilities. The multivariate probabilistic analyses can effectively consider the connection and interaction between drought characteristics, while conditional probability distribution allows analyzing the impact of one drought characteristic on another. The results also indicate a relatively high risk of streamflow drought with short duration and low severity in the region, requiring effective drought-mitigation strategies and measures.

Global-Local Dynamic Adversarial Learning for Cross-Domain Sentiment Analysis

As one of the most widely used applications in domain adaption (DA), Cross-domain sentiment analysis (CDSA) aims to tackle the barrier of lacking in sentiment labeled data. Applying an adversarial network to DA to reduce the distribution discrepancy between source and target domains is a significant advance in CDSA. This adversarial DA paradigm utilizes a single global domain discriminator or a series of local domain discriminators to reduce marginal or conditional probability distribution discrepancies. In general, each discrepancy has a different effect on domain adaption. However, the existing CDSA algorithms ignore this point. Therefore, in this paper, we propose an effective, novel and unsupervised adversarial DA paradigm, Global-Local Dynamic Adversarial Learning (GLDAL). This paradigm is able to quantitively evaluate the weights of global distribution and every local distribution. We also study how to apply GLDAL to CDSA. As GLDAL can effectively reduce the distribution discrepancy between domains, it performs well in a series of CDSA experiments and achieves improvements in classification accuracy compared to similar methods. The effectiveness of each component is demonstrated through ablation experiments on different parts and a quantitative analysis of the dynamic factor. Overall, this approach achieves the desired DA effect with domain shifts.

Multi-Modality Fusion & Inductive Knowledge Transfer Underlying Non-Sparse Multi-Kernel Learning and Distribution Adaption.

With the development of sensors, more and more multimodal data are accumulated, especially in biomedical and bioinformatics fields. Therefore, multimodal data analysis becomes very important and urgent. In this study, we combine multi-kernel learning and transfer learning, and propose a feature-level multi-modality fusion model with insufficient training samples. To be specific, we firstly extend kernel Ridge regression to its multi-kernel version under the lp-norm constraint to explore complementary patterns contained in multimodal data. Then we use marginal probability distribution adaption to minimize the distribution differences between the source domain and the target domain to solve the problem of insufficient training samples. Based on epilepsy EEG data provided by the University of Bonn, we construct 12 multi-modality & transfer scenarios to evaluate our model. Experimental results show that compared with baselines, our model performs better on most scenarios.

Maximum entropy copula for bivariate drought analysis

Copula functions are widely used to derive multivariate probability distributions in hydrometeorology. One of the key steps in the copula method is the derivation of marginal distributions of individual variables which can be accomplished using the principle of maximum entropy where the distribution parameters are estimated from the specified constraints. This study, investigated two drought variables (severity and duration) by coupling the principle of maximum entropy with parametric and empirical copulas. So, homogeneous climatic zones were first identified by applying the fuzzy clustering method to data from 39 synoptic stations in Iran and then drought severity and duration were determined with the standardized precipitation index. These two variables were scaled and their marginal probability distribution functions were derived using the principle of maximum entropy as well as empirically. Then, the joint probability distribution of drought severity and duration was determined using maximum entropy-copula, and parametric and empirical copulas. Thereafter, bivariate conditional return periods were determined for each homogeneous region. Results showed that 1) univariate and bivariate distributions can be obtained by maximizing entropy; 2) the dependence structure via Spearman's rho, which directly affects the Lagrange parameters of entropy copula, was a controlling factor to optimize the objective function; 3) for a given set of constraints, the maximum entropy copula is independent of the types of marginals; 4)The entropy-entropy copula (with entropy marginals) is considered a better method than alternatives, because it has a similar result to the parametric methods while it only needs to fit a single model.

Generalization of Cointegration Arbitrage Based on GARCH Model, Improved by Marginal Probability Distribution

As the Russia Ukraine War broke out in February, 2022, a series of international sanctions led to a panic among international hot moneys and slumps in global stock markets. For such a volatile market, hedging strategy is in a position to avoid risks. In finance, the correlation among financial derivative is always a topical issue and investors are concerned about the statistical methods to model the volatility. Recent years, the rapid development in data science makes it possible to record and manipulate massive data of financial derivative series, of which activates statistical arbitrage, a hedging strategy by pairs trading. The most widespread method in practice is cointegration arbitrage and many experiments show that it can always generate positive profits. This paper introduces the backgrounds of statistical arbitrage and relative definitions in cointegration arbitrage like time series, stationary, autocorrelation process, integration and cointegration. Based on the features of financial derivative series, this paper also illustrates error correction model, ARCH model and GARCH model. Finally, this paper shows the current strategy of cointegration arbitrage and raises some improvement by marginal probability distribution.

Property prediction for high-chromium high-vanadium steel based on transfer component analysis with few-shot guided

The transfer learning model improves accuracy by reducing the marginal and conditional probability distribution discrepancy between source and target domains. Based on the hypothesis of the ideal carbides of high-chromium high-vanadium steel, a mass fraction ratio written as (Cr + V)/C is deduced as vital feature to narrow the marginal probability distribution discrepancy. To align the conditional probability distribution of the source domain with the target domain, a few-shot guided transfer component analysis (TCA) method is proposed that a limited number of labeled samples taken from the target domain are used to guide the mapping. Then, the V/Cr combines with the optimal (Cr + V)/C is proposed to predict the composition of sample with the best wear resistance. Experimental results show that the proposed few-shot guided TCA method can considerably improve the prediction accuracy (R is higher than 0.99, RMSE is lower than 0.63HRC). The constructed (Cr + V)/C is the most critical feature. In addition, the predicted sample consisting of 2.1%C, 4%Cr, 4%V and 1.5%Mo has the best wear resistance with minimal abrasion weight loss in the test.

Geometric Brownian information engine: Essentials for the best performance.

We investigate a geometric Brownian information engine (GBIE) in the presence of an error-free feedback controller that transforms the information gathered on the state of Brownian particles entrapped in monolobal geometric confinement into extractable work. Outcomes of the information engine depend on the reference measurement distance x_{m}, the feedback site x_{f}, and the transverse force G. We determine the benchmarks for utilizing the available information in an output work and the optimum operating requisites for best achievable work. Transverse bias force (G) tunes the entropic contribution in the effective potential and hence the standard deviation (σ) of the equilibrium marginal probability distribution. We recognize that the amount of extractable work reaches a global maximum when x_{f}=2x_{m} with x_{m}∼0.6σ, irrespective of the extent of the entropic limitation. Because of the higher loss of information during the relaxation process, the best achievable work of a GBIE is lower in an entropic system. The feedback regulation also bears the unidirectional passage of particles. The average displacement increases with growing entropic control and is maximum when x_{m}∼0.81σ. Finally, we explore the efficacy of the information engine, a quantity that regulates the efficiency in utilizing the information acquired. With x_{f}=2x_{m}, the maximum efficacy reduces with increasing entropic control and shows a crossover from 2 to 11/9. We discover that the condition for the best efficacy depends only on the confinement lengthscale along the feedback direction. The broader marginal probability distribution accredits the increased average displacement in a cycle and the lower efficacy in an entropy-dominated system.

Outlier-Robust Surrogate Modeling of Ion-Solid Interaction Simulations.

Data for complex plasma-wall interactions require long-running and expensive computer simulations. Furthermore, the number of input parameters is large, which results in low coverage of the (physical) parameter space. Unpredictable occasions of outliers create a need to conduct the exploration of this multi-dimensional space using robust analysis tools. We restate the Gaussian process (GP) method as a Bayesian adaptive exploration method for establishing surrogate surfaces in the variables of interest. On this basis, we expand the analysis by the Student-t process (TP) method in order to improve the robustness of the result with respect to outliers. The most obvious difference between both methods shows up in the marginal likelihood for the hyperparameters of the covariance function, where the TP method features a broader marginal probability distribution in the presence of outliers. Eventually, we provide first investigations, with a mixture likelihood of two Gaussians within a Gaussian process ansatz for describing either outlier or non-outlier behavior. The parameters of the two Gaussians are set such that the mixture likelihood resembles the shape of a Student-t likelihood.

Deep Gaussian mixture adaptive network for robust soft sensor modeling with a closed-loop calibration mechanism

Process drift leads to an out-of-distribution problem between historical training data and online deployment data, which deteriorates the performance of soft sensors. Unfortunately, few existing deep learning soft sensors have calibration mechanisms to deal with the process drift. To this end, this article proposes a robust Deep Gaussian Mixture Adaptive Network (DGMAN)-based soft sensor with a closed-loop calibration mechanism. First, a Gaussian Mixture Conditional Variational Autoencoder (GMCVAE) is presented, with the ability to capture the multimodality in industrial data. Afterwards, we propose a novel semi-supervised Gaussian mixture domain adaptation to perform conditional and marginal probability distribution alignment in the Gaussian mixture probabilistic latent space created by GMCVAE, using both labeled and unlabeled target samples. Theoretically, two kinds of process drift, including concept drift and virtual drift, could be alleviated by conditional distribution adaptation and marginal distribution adaptation, respectively. In addition, a new closed-loop calibration mechanism is deployed in a pre-training and fine-tuning fashion. Finally, robust analysis in theory and experimental results on the gas turbine dataset demonstrate that the proposed DGMAN-based soft sensor with robustness against process drift can maintain long-term validity in real industrial processes. Calibrated by only 100 labeled samples per year, the RMSE of DGMAN is 5.55 ± 0.32 after three years of deployment in the power plant, outperforming all comparison methods.

Analyzing Extreme Sea State Conditions by Time-Series Simulation Accounting for Seasonality

AbstractThis article presents an extreme value analysis on data of significant wave height based on time-series simulation. A method to simulate time series with given marginal distribution and preserving the autocorrelation structure in the data is applied to significant wave height data. Then, extreme value analysis is performed by simulating from the fitted time-series model that preserves both the marginal probability distribution and the autocorrelation. In this way, the effect of serial correlation on the extreme values can be taken into account, without subsampling and de-clustering of the data. The effect of serial correlation on estimating extreme wave conditions have previously been highlighted, and failure to account for this effect will typically lead to an overestimation of extreme conditions. This is demonstrated by this study, which compares extreme value estimates from the simulated times-series model with estimates obtained directly from the marginal distribution assuming that 3-h significant wave heights are independent and identically distributed. A dataset of significant wave height provided as part of a second benchmark exercise on environmental extremes that was presented at OMAE 2021 has been analyzed. This article is an extension of a study presented at OMAE 2022 (OMAE2022-78795) and includes additional preprocessing of the data to account for seasonality and new results.

Reexamination of the Kochen-Specker theorem: Relaxation of the completeness assumption

The Kochen-Specker theorem states that exclusive and complete deterministic outcome assignments are impossible for certain sets of measurements, called Kochen-Specker (KS) sets. A straightforward consequence is that KS sets do not have joint probability distributions because no set of joint outcomes over such a distribution can be constructed. However, we show it is possible to construct a joint quasiprobability distribution over any KS set by relaxing the completeness assumption. Interestingly, completeness is still observable at the level of measurable marginal probability distributions. This suggests the observable completeness might not be a fundamental feature, but a secondary property.

The Bayesian Estimate of Vector Autoregressive Model Parameters Adopt Informative Prior Information

This research included the bayesian estimate for vector Autoregressive model with rank (p) in addition to statistical tests and predict Bayesian when the random error of model followed generalized multivariate modified Bessel distribution. The prior information about the parameters of model is represented by probability distributions belong to conjugate families. It found that the posterior marginal probability distribution for parameters matrix (Φ) is a Matrix t distribution, The posterior marginal probability distribution of covariance matrix (Σ) is uncommon and the predictive probability distribution of future observations vector is multivariate t distribution.