non-Gaussian Data Research Articles

Bayesian estimation in generalized linear models for longitudinal data with hyperspherical coordinates

Under the framework of generalized linear models (GLM), the generalized estimating equation (GEE) method is typically applied for longitudinal data analysis. However, there are a series of problems due to the misspecification of the within-subject correlation structure, especially in Bayesian estimation. To handle these difficulties, in this paper, we construct a class of generalized estimating equations for longitudinal data with hyperspherical coordinates (HPC) and propose a Bayesian approach established through empirical likelihood (EL). Additionally, an efficient Markov chain Monte Carlo (MCMC) procedure is developed for the required computation of the posterior distribution. As proved by the simulation studies and an application to a real longitudinal data set, our method not only performs better than traditional empirical likelihood estimation and Bayesian estimation with partial autocorrelations (PAC) but also is suitable for non-Gaussian data.

Adaptive and fuzzy locality discriminant analysis for dimensionality reduction

Linear discriminant analysis (LDA) uses labeled samples for acquiring a discriminant projection direction, by which data of different categories are separated into distinct groups in a lower-dimensional subspace. In response to the issue that LDA lacks robustness to non-Gaussian data with rich local information, improvements on LDA explore the subspace manifold structure by building a fully connected similarity graph. However, these methods are vulnerable to the interference of noisy and redundant information. In this paper, we propose a new local LDA method, named adaptive and fuzzy locality discriminant analysis (AFLDA), which aims at extracting precise and compact features. Firstly, an adaptive and fuzzy k-means strategy is adopted, where the membership between data and corresponding subclass centers for each class is learned to establish a hybrid bipartite graph and capture local information. Secondly, we design a discrete and probability constraint imposed on the membership matrix to explore the intricate structure of multimodal data. Moreover, the subblock partition for each class makes data accommodate to multimodal subclasses. Thirdly, the maximum total scatter regularization term is introduced, which amply disperses the data to enhance the recognition of local structure and avoid trivial solutions. Finally, to eliminate the interference of noisy and redundant information, AFLDA learns an optimized subspace, where cluster centers and the membership matrix are updated alternately. Promising results in experiments illustrate the efficacy of the model.

Influence of α-Stable Noise on the Effectiveness of Non-Negative Matrix Factorization—Simulations and Real Data Analysis

Non-negative matrix factorization (NMF) has been used in various applications, including local damage detection in rotating machines. Recent studies highlight the limitations of diagnostic techniques in the presence of non-Gaussian noise. The authors examine the impact of non-Gaussianity levels on the extraction of the signal of interest (SOI). The simple additive model of the signal is proposed: SOI and non-Gaussian noise. As a model of the random component, i.e., noise, a heavy-tailed α-stable distribution with two important parameters (σ and α) was proposed. If SOI is masked by noise (controlled by σ), the influence of non-Gaussianity level (controlled by α) is more critical. We performed an empirical analysis of how these parameters affect SOI extraction effectiveness using NMF. Finally, we applied two NMF algorithms to several (both vibration and acoustic) signals from a machine with faulty bearings at different levels of non-Gaussian disturbances and the obtained results align with the simulations. The main conclusion of this study is that NMF is a very powerful tool for analyzing non-Gaussian data and can provide satisfactory results in a wide range of a non-Gaussian noise levels.

Gap-MK-DCCA-Based Intelligent Fault Diagnosis for Nonlinear Dynamic Systems

In intelligent process monitoring and fault detection of the modern process industry, conventional methods mostly consider singular characteristics of systems. To tackle the problem of suboptimal incipient fault detection in nonlinear dynamic systems with non-Gaussian distributed data, this paper proposes a methodology named Gap-Mixed Kernel-Dynamic Canonical Correlation Analysis. Initially, the Gap metric is employed for data preprocessing, followed by fault detection utilizing the Mixed Kernel-Dynamic Canonical Correlation Analysis. Ultimately, fault identification is conducted through a contribution method based on the T2 statistic. Furthermore, a comparative analysis was conducted using Canonical Variate Analysis, Dynamic Canonical Correlation Analysis, and Mixed Kernel-Dynamic Canonical Correlation Analysis on the Tennessee Eastman process. Experimental results indicate varying degrees of improvements in the detection rate, false alarm rate, missed detection rate, and detection time compared to the comparative methods, demonstrating the industrial value and academic significance of the method.

A parsimonious, computationally efficient machine learning method for spatial regression

AbstractWe introduce the modified planar rotator method (MPRS), a physically inspired machine learning method for spatial/temporal regression. MPRS is a non-parametric model which incorporates spatial or temporal correlations via short-range, distance-dependent “interactions” without assuming a specific form for the underlying probability distribution. Predictions are obtained by means of a fully autonomous learning algorithm which employs equilibrium conditional Monte Carlo simulations. MPRS is able to handle scattered data and arbitrary spatial dimensions. We report tests on various synthetic and real-word data in one, two and three dimensions which demonstrate that the MPRS prediction performance (without hyperparameter tuning) is competitive with standard interpolation methods such as ordinary kriging and inverse distance weighting. MPRS is a particularly effective gap-filling method for rough and non-Gaussian data (e.g., daily precipitation time series). MPRS shows superior computational efficiency and scalability for large samples. Massive datasets involving millions of nodes can be processed in a few seconds on a standard personal computer. We also present evidence that MPRS, by avoiding the Gaussian assumption, provides more reliable prediction intervals than kriging for highly skewed distributions.

Application of robust regression in translational neuroscience studies with non-Gaussian outcome data.

Linear regression is one of the most used statistical techniques in neuroscience, including the study of the neuropathology of Alzheimer's disease (AD) dementia. However, the practical utility of this approach is often limited because dependent variables are often highly skewed and fail to meet the assumption of normality. Applying linear regression analyses to highly skewed datasets can generate imprecise results, which lead to erroneous estimates derived from statistical models. Furthermore, the presence of outliers can introduce unwanted bias, which affect estimates derived from linear regression models. Although a variety of data transformations can be utilized to mitigate these problems, these approaches are also associated with various caveats. By contrast, a robust regression approach does not impose distributional assumptions on data allowing for results to be interpreted in a similar manner to that derived using a linear regression analysis. Here, we demonstrate the utility of applying robust regression to the analysis of data derived from studies of human brain neurodegeneration where the error distribution of a dependent variable does not meet the assumption of normality. We show that the application of a robust regression approach to two independent published human clinical neuropathologic data sets provides reliable estimates of associations. We also demonstrate that results from a linear regression analysis can be biased if the dependent variable is significantly skewed, further indicating robust regression as a suitable alternate approach.

DeepKriging: Spatially Dependent Deep Neural Networks for Spatial Prediction

In spatial statistics, a common objective is to predict values of a spatial process at unobserved locations by exploiting spatial dependence. Kriging provides the best linear unbiased predictor using covariance functions and is often associated with Gaussian processes. However, when considering non-linear prediction for non-Gaussian and categorical data, the Kriging prediction is no longer optimal, and the associated variance is often overly optimistic. Although deep neural networks (DNNs) are widely used for general classification and prediction, they have not been studied thoroughly for data with spatial dependence. In this work, we propose a novel DNN structure for spatial prediction, where the spatial dependence is captured by adding an embedding layer of spatial coordinates with basis functions. We show in theory and simulation studies that the proposed DeepKriging method has a direct link to Kriging in the Gaussian case, and it has multiple advantages over Kriging for non-Gaussian and non-stationary data, i.e., it provides non-linear predictions and thus has smaller approximation errors, it does not require operations on covariance matrices and thus is scalable for large datasets, and with sufficiently many hidden neurons, it provides the optimal prediction in terms of model capacity. We further explore the possibility of quantifying prediction uncertainties based on density prediction without assuming any data distribution. Finally, we apply the method to predicting PM2.5 concentrations across the continental United States.

A double regression method for graphical modeling of high-dimensional nonlinear and non-Gaussian data

A double regression method for graphical modeling of high-dimensional nonlinear and non-Gaussian data

Testing Conditional Independence Between Latent Variables by Independence Residuals.

Conditional independence (CI) testing is an important problem, especially in causal discovery. Most testing methods assume that all variables are fully observable and then test the CI among the observed data. Such an assumption is often untenable beyond applications dealing with, e.g., psychological analysis about the mental health status and medical diagnosing (researchers need to consider the existence of latent variables in these scenarios); and typically adopted latent CI test schemes mainly suffer from robust or efficient issues. Accordingly, this article investigates the problem of testing CI between latent variables. To this end, we offer an auxiliary regression-based CI (AReCI) test by taking the measured variable as the surrogate variable of the latent variables to conduct the regression over the latent variables under the linear causal models, in which each latent variable has some certain measured variables. Specifically, given a pair of latent variables LX and LY , and a corresponding latent variable set LO , [Formula: see text] holds if and only if [Formula: see text] and [Formula: see text] are statistically independent, where A' and A'' are the two disjoint subset of the measured variable for the corresponding latent variables, A'{LO} ∩A''{LO} = ∅ , and ω1 is a parameter vector characterized from the cross covariance between A{LX} and A'{LO} , and ω2 is a parameter vector characterized from the cross covariance between A{LY} and A''{LO} . We theoretically show that the AReCI test is capable of addressing both Gaussian and non-Gaussian data. In addition, we find that the well-known partial correlation test can be seen as a special case of the AReCI test. Finally, we devise a causal discovery method by using the AReCI test as the CI test. The experimental results on synthetic and real-world data illustrate the effectiveness of our method.

Application of maximum statistical entropy in formulating a non-gaussian probability density function in flow uncertainty analysis with prior measurement knowledge

In mechanical, civil and chemical engineering systems the accuracies of flow measurement instruments is conventionally specified by certified measurement capabilities (CMCs) that are symmetric, however it is physically possible for some flow instruments and equipment to exhibit asymmetric non-Gaussian behaviour. In this paper the influence of non-Gaussian uncertainties is investigated using direct Monte Carlo simulations to construct a probability density function (PDF) using representative non-Gaussian surface roughness data for a commercial steel pipe friction factor. Actual PDF results are compared and contrasted with a symmetric Gaussian PDF, and reveal inconsistencies in the statistical distributions that cannot be neglected in high accuracy flow measurements. The non-Gaussian PDF is visualized with a kernel density estimate (KDE) scheme to infer an initial qualitative shape of the actual PDF using the approximate locations of the normalized peaks as a initial metrologist estimate of the measurement density. This is then utilized as inputs in a maximum statistical entropy functional to optimize the actual non-Gaussian PDF using a nonlinear optimization of Lagrange multipliers for a mathematically unique PDE. Novelties in the present study is that a new methodology has been developed for statistical sampling from non-monotonic non-Gaussian distributions with accompanying Python and Matlab/GNU Octave computer codes, and a new methodology for utilizing metrologist's expert prior knowledge of PDF peaks and locations for constructing an a priori estimate of the shape of unknown density have been incorporated into the maximum statistical entropy nonlinear optimization problem for a faster and more efficient approach for generating statistical information and insights in constructing high accuracy non-Gaussian PDFs of real world messy engineering measurements.

Prediction and reliability analysis of reservoir lithology spatial distribution

Reliable lithology spatial distribution directly reflects the geological situation of the reservoir, which is the basis of stratigraphic correlation, sedimentary modeling, and other geological research. Under the condition of limited reservoir data, it is a challenging task to accurately depict the lithology spatial distribution and provide a quantitative reliability analysis of the results. In this study, we propose a flexible spatial distribution prediction and model reliability analysis method. Firstly, the method develops a spatially dependent deep Kriging technology to fit the heterogeneous characteristics of the reservoir lithology, and adopts the extracted spatial key information and related reservoir attributes to invert lithology spatial distribution intelligently. Then, it focuses on the real-time assimilation of non-Gaussian data in the reliability modeling and quantitatively analyzes the reliability of the prediction system under the non-Gaussian hypothesis. Finally, the method is applied to the actual heterogeneous reservoir, good results are achieved in the prediction accuracy, model fitting degree, model reliability, and time performance compared with other methods. The method is conducive to finding future mineral deposits locations and reducing exploration costs.

A novel degradation model and reliability evaluation methodology based on two-phase feature extraction: An application to marine lubricating oil pump

The Wiener-process-based Degradation Approach with Two-phase Algorithm (W-DATA) is introduced as a multi-dimensional framework explicitly designed for modeling mechanical system reliability. This approach leverages an enhanced Empirical Mode Decomposition, known as P-EMD, for degradation feature extraction, and utilizes Approximate Entropy (ApEn) for quantifying feature complexities. W-DATA seamlessly integrates these components into a time-resolved degradation model. To enhance its robustness, the proposed reliability model synergizes with a Box-Cox transformed Wiener model, allowing for improved adaptability to non-Gaussian data landscapes. The paper outlines a strategy for establishing a reliable threshold for this data-driven degradation model based on non-Gaussian data, a critical aspect for reliability modeling using the Wiener process. This strategy addresses the challenge of Wiener process verification through normality and independent testing of data increments. The integrative methodology is recommended for application in machinery reliability analytics and has undergone rigorous validation through a 770 h real-world test on a marine lubricating oil pump reducer. The results affirm its efficacy and feasibility in accurately estimating mechanical system reliability.

Local Embedding Learning via Landmark-Based Dynamic Connections.

Linear discriminant analysis (LDA) is one of the most effective and popular methods to reduce the dimensionality of data with Gaussian assumption. However, LDA cannot handle non-Gaussian data because the center point is incompetent to represent the distribution of data. Some existing methods based on graph embedding focus on exploring local structures via pairwise relationships of data for addressing the non-Gaussian issue. Due to massive pairwise relationships, the computational complexity is high as well as the locally optimal solution is hard to find. To address these issues, we propose a novel and efficient local embedding learning via landmark-based dynamic connections (LDC) in which we leverage several landmarks to represent different subclusters in the same class and establish the connections between each point and landmark. Furthermore, in order to explore the relationship of landmarks pairwise more precisely, the relationship between each point and their corresponding neighbor landmarks are found in the optimal subspace, rather than the original space, which can avoid the negative influence of the noises. We also propose an efficient iterative algorithm to deal with the proposed ratio minimization problem. Extensive experiments conducted on several real-world datasets have demonstrated the advantages of the proposed method.

Bispectral analysis and information fusion technique for bearing fault classification

The feasibility and effectiveness of data fusion for the fault classification of bearing faults have been very well iterated in the literature. However, all previous endeavors have been limited to time, frequency, and time-frequency domain techniques. The use of higher-order spectral analysis (HOSA), especially Bispectrum and Trispectrum, for fault detection is gaining importance in recent studies due to the many advantages of HOSA. Bispectral features provide a valuable tool for capturing higher-order statistical relationships in signals, making them particularly effective in detecting nonlinearities and distinguishing between Gaussian and non-Gaussian data. Their robustness to noise and ability to reveal hidden information make them advantageous in applications such as vibration analysis, speech recognition, and image processing, where complex signal interactions and nonlinearity play a significant role in data interpretation and pattern recognition. This paper proposes a methodology for the fusion of the data from the vibration and the acoustic sensors for the fault detection of roller element bearings using bispectral features. Higher-order spectral characteristics are derived from vibration and acoustic sensor data, and they are fused using artificial neural networks and various other machine learning algorithms like support vector machine, K nearest neighbor, Naïve Bayes algorithm, and decision tree. This work primarily aims to evaluate the performance of each classifier when applied to the fused data, in contrast to the performance when using individual sensor data alone. The outcomes revealed that, even though the accuracy of the acoustic sensor data was lower in comparison to the vibration sensor data, which exhibited the highest performance of 100% accuracy with nearly all the classifiers, the fused data achieved remarkable results of 100% accuracy with artificial neural networks and decision trees. However, the Naïve Bayes algorithm yielded the lowest accuracy when applied to the fused data. The primary objective of this paper is to demonstrate the application of bispectrum analysis for data fusion and to enhance confidence in fault detection. It achieves this by maintaining the capability to accurately and dependably detect faults, even when a single sensor encounters issues or falls short of anticipated performance standards.

A new reconstruction-based method using local Mahalanobis distance for incipient fault isolation and amplitude estimation

Faulty variable isolation and amplitude estimation are of great importance to support the decision-making for system maintenance but lack sufficient studies, especially concerning the challenge of incipient faults with tiny amplitude. The reconstruction-based contribution (RBC) idea is commonly used for faulty variable isolation and fault amplitude estimation but usually suffers from low accuracy performance when facing the incipient faults challenge due to the use of insensitive detection indexes. Therefore, this paper proposes an improved reconstruction-based approach using the highly sensitive index named local Mahalanobis distance (LMD) for incipient fault isolation and amplitude estimation. The novel RBC approach retains the advantages of LMD, e.g., high sensitivity to incipient faults, robustness to outliers, and ability to handle non-Gaussian data, and is also available for multiple faulty variables isolation. The performance evaluation of the proposed methods using the benchmark case of the Continuous-flow Stirred Tank Reactor (CSTR) process shows that this approach has high isolation and estimation accuracy for both single and multiple faults. Thanks to a comparative study, it can be highlighted that for both faulty variables isolation and fault amplitude estimation tasks, our proposal outperforms the state-of-the-art RBC-based methods using different detection indexes: the combined index of PCA, conventional Mahalanobis distance, and augmented Mahalanobis distance.

Unsupervised Mixture Models on the Edge for Smart Energy Consumption Segmentation with Feature Saliency.

Smart meter datasets have recently transitioned from monthly intervals to one-second granularity, yielding invaluable insights for diverse metering functions. Clustering analysis, a fundamental data mining technique, is extensively applied to discern unique energy consumption patterns. However, the advent of high-resolution smart meter data brings forth formidable challenges, including non-Gaussian data distributions, unknown cluster counts, and varying feature importance within high-dimensional spaces. This article introduces an innovative learning framework integrating the expectation-maximization algorithm with the minimum message length criterion. This unified approach enables concurrent feature and model selection, finely tuned for the proposed bounded asymmetric generalized Gaussian mixture model with feature saliency. Our experiments aim to replicate an efficient smart meter data analysis scenario by incorporating three distinct feature extraction methods. We rigorously validate the clustering efficacy of our proposed algorithm against several state-of-the-art approaches, employing diverse performance metrics across synthetic and real smart meter datasets. The clusters that we identify effectively highlight variations in residential energy consumption, furnishing utility companies with actionable insights for targeted demand reduction efforts. Moreover, we demonstrate our method's robustness and real-world applicability by harnessing Concordia's High-Performance Computing infrastructure. This facilitates efficient energy pattern characterization, particularly within smart meter environments involving edge cloud computing. Finally, we emphasize that our proposed mixture model outperforms three other models in this paper's comparative study. We achieve superior performance compared to the non-bounded variant of the proposed mixture model by an average percentage improvement of 7.828%.

Test of conditional independence in factor models via Hilbert–Schmidt independence criterion

This work is concerned with testing conditional independence under a factor model setting. We propose a novel multivariate test for non-Gaussian data based on the Hilbert–Schmidt independence criterion (HSIC). Theoretically, we investigate the convergence of our test statistic under both the null and the alternative hypotheses, and devise a bootstrap scheme to approximate its null distribution, showing that its consistency is justified. Methodologically, we generalize the HSIC-based independence test approach to a situation where data follow a factor model structure. Our test requires no nonparametric smoothing estimation of functional forms including conditional probability density functions, conditional cumulative distribution functions and conditional characteristic functions under the null or alternative, is computationally efficient and is dimension-free in the sense that the dimension of the conditioning variable is allowed to be large but finite. Further extension to nonlinear, non-Gaussian structure equation models is also described in detail and asymptotic properties are rigorously justified. Numerical studies demonstrate the effectiveness of our proposed test relative to that of several existing tests.

Novel reduced kernel independent component analysis for process monitoring

Kernel independent component analysis (KICA), as a nonlinear extension monitoring method of independent component analysis (ICA), has attracted significant attention. To accomplish different monitoring tasks for nonlinear systems with non-Gaussian data distribution, many modified algorithms based on KICA have also been designed. However, most of the existing methods suffer from defects; for example, the computation time increases with the number of training samples and the models are insensitive to minor faults. Nevertheless, there is currently limited research on addressing these defects, which greatly limits their application in industrial processes. To fill these gaps, a novel reduced kernel independent component analysis (NRKICA) method is proposed to reduce the computation complexity and improve the ability of minor fault detection at the same time. In this approach, an important factor is defined to measure the ability of the samples to represent the properties of the system. In addition, then the top-n important observations are selected to build a data dictionary. To improve the sensitivity to minor faults, the [Formula: see text] and [Formula: see text] statistics are redesigned by introducing information from past observations. Besides, the kernel parameter is optimized by the tabu search algorithm. The proposed method is applied to fault detection with a numerical example and the Tennessee Eastman process (TEP), and the experimental results verify the effectiveness and sensitivity of the proposed method.

BG2: Bayesian variable selection in generalized linear mixed models with nonlocal priors for non-Gaussian GWAS data

BackgroundGenome-wide association studies (GWASes) aim to identify single nucleotide polymorphisms (SNPs) associated with a given phenotype. A common approach for the analysis of GWAS is single marker analysis (SMA) based on linear mixed models (LMMs). However, LMM-based SMA usually yields a large number of false discoveries and cannot be directly applied to non-Gaussian phenotypes such as count data.ResultsWe present a novel Bayesian method to find SNPs associated with non-Gaussian phenotypes. To that end, we use generalized linear mixed models (GLMMs) and, thus, call our method Bayesian GLMMs for GWAS (BG2). To deal with the high dimensionality of GWAS analysis, we propose novel nonlocal priors specifically tailored for GLMMs. In addition, we develop related fast approximate Bayesian computations. BG2 uses a two-step procedure: first, BG2 screens for candidate SNPs; second, BG2 performs model selection that considers all screened candidate SNPs as possible regressors. A simulation study shows favorable performance of BG2 when compared to GLMM-based SMA. We illustrate the usefulness and flexibility of BG2 with three case studies on cocaine dependence (binary data), alcohol consumption (count data), and number of root-like structures in a model plant (count data).

A novel uncertainty evaluation method based on the particle filter and beta distribution for data with unknown distribution.

Uncertainty evaluation for unknown distribution data is a key problem to be solved in uncertainty evaluation theory. To evaluate the measurement uncertainty of data with unknown distributions, a novel uncertainty evaluation method based on the particle filter (PF) and beta distribution is proposed in this paper. A beta distribution with wide adaptability was adopted as the distribution type of measurement results, the parameters of the beta distribution were taken as the parameters to be estimated, and a state-space model was established. The PF method, suitable for non-Gaussian data, was utilized to obtain the estimates of the parameters of the beta distribution according to the measurement results. Finally, the best estimates of the measurement results and their uncertainty were calculated using the beta distribution parameters. Simulation results show that the proposed method is adaptive to accurately evaluate the measurement uncertainties of data, especially for non-Gaussian distribution data or asymmetrically distributed data. Multiple evaluation results show that the method has good robustness. The experimental results for the drift errors of a laser interferometer show that the uncertainty result of the proposed method is consistent with the Monte Carlo method. This method is suitable for a variety of distribution types that can be characterized through beta distribution and can solve the optimal estimation and uncertainty evaluation of most measurement results with unknown distribution types.