Higher Order Statistical Information Research Articles

EHOPN: A novel enhanced high-order pooling-based network for industrial fault detection

Recently, deep learning algorithms have been successfully applied to industrial fault detection because they are better at automatically extracting complex features and processing high-dimensional data than traditional methods. However, most existing deep learning-based fault detection methods only concentrate on extracting features from industrial process data without considering the crucial long-term temporal features and higher-order statistical information. To address this challenge, we proposed a novel enhanced higher-order pooling-based network (EHOPN) for industrial fault detection. First, the data pre-processing of the network is presented to capture the dynamic features of time-series process data and unify the high-dimensional data scale. Second, the EHOPN utilizes channel and temporal second-order pooling techniques to gather temporal and channel statistics information, facilitating the backbone network’s ability to capture complex inter-dependencies and long-term dynamics. Additionally, the high-order feature aggregation module is presented to aggregate global and local features, enhancing the network’s generalization ability. The proposed industrial fault detection approach is evaluated on the Tennessee Eastman benchmark and a real-world heavy-plate production process. Experimental results show that the proposed method is significantly better than comparison models in four evaluation metrics: accuracy, precision, recall, and F1-score, further proving the effectiveness of EHOPN.

Category-Level Object Pose Estimation with Statistic Attention.

Six-dimensional object pose estimation is a fundamental problem in the field of computer vision. Recently, category-level object pose estimation methods based on 3D-GC have made significant breakthroughs due to advancements in 3D-GC. However, current methods often fail to capture long-range dependencies, which are crucial for modeling complex and occluded object shapes. Additionally, discerning detailed differences between different objects is essential. Some existing methods utilize self-attention mechanisms or Transformer encoder-decoder structures to address the lack of long-range dependencies, but they only focus on first-order information of features, failing to explore more complex information and neglecting detailed differences between objects. In this paper, we propose SAPENet, which follows the 3D-GC architecture but replaces the 3D-GC in the encoder part with HS-layer to extract features and incorporates statistical attention to compute higher-order statistical information. Additionally, three sub-modules are designed for pose regression, point cloud reconstruction, and bounding box voting. The pose regression module also integrates statistical attention to leverage higher-order statistical information for modeling geometric relationships and aiding regression. Experiments demonstrate that our method achieves outstanding performance, attaining an mAP of 49.5 on the 5°2 cm metric, which is 3.4 higher than the baseline model. Our method achieves state-of-the-art (SOTA) performance on the REAL275 dataset.

HHOMR: a hybrid high-order moment residual model for miRNA-disease association prediction.

Numerous studies have demonstrated that microRNAs (miRNAs) are critically important for the prediction, diagnosis, and characterization of diseases. However, identifying miRNA-disease associations through traditional biological experiments is both costly and time-consuming. To further explore these associations, we proposed a model based on hybrid high-order moments combined with element-level attention mechanisms (HHOMR). This model innovatively fused hybrid higher-order statistical information along with structural and community information. Specifically, we first constructed a heterogeneous graph based on existing associations between miRNAs and diseases. HHOMR employs a structural fusion layer to capture structure-level embeddings and leverages a hybrid high-order moments encoder layer to enhance features. Element-level attention mechanisms are then used to adaptively integrate the features of these hybrid moments. Finally, a multi-layer perceptron is utilized to calculate the association scores between miRNAs and diseases. Through five-fold cross-validation on HMDD v2.0, we achieved a mean AUC of 93.28%. Compared with four state-of-the-art models, HHOMR exhibited superior performance. Additionally, case studies on three diseases-esophageal neoplasms, lymphoma, and prostate neoplasms-were conducted. Among the top 50 miRNAs with high disease association scores, 46, 47, and 45 associated with these diseases were confirmed by the dbDEMC and miR2Disease databases, respectively. Our results demonstrate that HHOMR not only outperforms existing models but also shows significant potential in predicting miRNA-disease associations.

Constrained Independent Vector Analysis with Reference for Multi-Subject fMRI Analysis.

Independent component analysis (ICA) is now a widely used solution for the analysis of multi-subject functional magnetic resonance imaging (fMRI) data. Independent vector analysis (IVA) generalizes ICA to multiple datasets (multi-subject data). Along with higher-order statistical information in ICA, it leverages the statistical dependence across the datasets as an additional type of statistical diversity. As such, IVA preserves variability in the estimation of single-subject maps but its performance might suffer when the number of datasets increases. Constrained IVA is an effective way to bypass computational issues and improve the quality of separation by incorporating available prior information. Existing constrained IVA approaches often rely on user-defined threshold values to define the constraints. However, an improperly selected threshold can have a negative impact on the final results. This paper proposes two novel methods for constrained IVA: one using an adaptive-reverse scheme to select variable thresholds for the constraints and a second one based on a threshold-free formulation by leveraging the unique structure of IVA. Notably, the proposed algorithms do not require all components to be constrained, utilizing free components to model interferences and components that might not be in the reference set. We demonstrate that our solutions provide an attractive solution to multi-subject fMRI analysis both by simulations and through analysis of resting state fMRI data collected from 98 subjects - the highest number of subjects ever used by IVA algorithms. Our results show that both proposed approaches obtain significantly better separation quality and model match while providing computationally efficient and highly reproducible solutions.

MFMANet: Multi-feature Multi-attention Network for efficient subtype classification on non-small cell lung cancer CT images

Lung cancer is one of the most prevalent diseases worldwide, being the most common type of cancer. Non-small cell lung cancer (NSCLC) with a five-year survival rate of less than 20% and widely varying treatment modalities is diagnosed in more than 85% of cases. In this study, a multi-feature multi-attention network (MFMANet) was proposed for NSCLC subtype classification to address the problem of low classification accuracy between subtypes due to small lesion size and similar background. In MFMANet, the multi-scale spatial channel attention module (MSAM) and multi-feature fusion global local attention module (MFGLA), are proposed. MSAM is able to preserve spatial features during channel feature fusion to capture high-order statistical information. MFGLA performs effective fusion of multiple features to avoid interference caused by scale differences. The global and local information is extracted by global and local attention branches to enhance the perception of small lesion regions. The performance of the proposed MFMANet was validated on two public datasets and compared with other classification networks including ResNet18, ShuffleNetv2, MobileNetv3, and MnasNet. MFMANet achieved 99.06% and 91.67% accuracy in the binary classification of CT images of lung adenocarcinoma (ADC) and lung squamous cell carcinoma (SCC), outperforming other methods. This study confirms that the proposed MFMANet provides effective solution for the subtypes classification of NSCLC.

A new framework for the reconstruction of porous media based on statistical characteristics: Multiscale erosion simulated annealing method

In this paper, a multiscale erosion simulated annealing method (MESA) is proposed, which can improve the computational efficiency of microstructure reconstruction by an order of magnitude. The reconstruction is performed on two-dimensional (2D) and three-dimensional (3D) microstructure based on simulated annealing (SA) and MESA, where the two-point correlation function is selected as the morphological information descriptor. The two-point cluster function containing significant high-order statistical information is used to verify the reconstruction results. From the analysis of 2D reconstruction, it can be found that although the proposed MESA technology can improve the reconstruction efficiency, the reconstruction is still not perfect. That is because the two-point correlation function contains insufficient information. In the 3D reconstruction, the two-point correlation function of the reconstructed 3D microstructure has a good consistency with the original 2D two-point correlation function, indicating that our proposed method is effective for the 3D microstructure reconstruction. By comparing the energy and computation time of SA and MESA, it is shown that our proposed method is one order of magnitude faster than SA method. This is because only a small part of pixels in the overall scales need to be sampled.

Reconstruction of 3D multi-mineral shale digital rock from a 2D image based on multi-point statistics

Introduction: Shale oil and gas reservoirs contain a variety of inorganic and organic pores that differ significantly from conventional reservoirs, making traditional experiments ineffective. Instead, the pore-scale imaging and modeling method, regarded as a novel and practical approach, is proposed to characterize shale microstructure and petrophysical properties. Therefore, it is of great significance to accurately reconstruct the three-dimensional (3D) microstructure of the porous medium, that is, the digital rock. However, microstructural images of shale at high-resolution, obtained through scanning electron microscopy (SEM) are constrained in the two-dimensional (2D) scale.Method: In this work, a novel iterative algorithm to reconstruct 3D multi-phase shale digital rock from a 2D image using multi-point statistics has been proposed. A multi-grid data template was used to capture the conditional probabilities and data events. The novelty of this work stems from an accurate representation of different types of pores and the mineral characteristics of shale rock from 2D images.Result: A series of simulations were conducted to reconstruct 2D shale digital rock from a 2D segmented training image, 3D shale digital rock from a 2D segmented training image, a 2D gray training image to reconstruct 2D shale digital rock, and a 2D gray training image to reconstruct 3D shale digital rock.Discussion: To corroborate the accuracy of the reconstructed digital rock and evaluate the reliability of the proposed algorithm, we compared the construction image with the training image with the two-point correlation function, geometry, morphological topology structure, and flow characteristics. The reconstruction accuracy indicates that the proposed algorithm can replicate the higher-order statistical information of the training image.

On the seasonal prediction and predictability of winter surface Temperature Swing Index over North America

The rapid day-to-day temperature swings associated with extratropical storm tracks can cause cascading infrastructure failure and impact human outdoor activities, thus research on seasonal prediction and predictability of extreme temperature swings is of huge societal importance. To measure the extreme surface air temperature (SAT) variations associated with the winter extratropical storm tracks, a Temperature Swing Index (TSI) is formulated as the standard deviation of 24-h-difference-filtered data of the 6-hourly SAT. The dominant term governing the TSI variability is shown to be proportional to the product of eddy heat flux and mean temperature gradient. The seasonal prediction skill of the winter TSI over North America was assessed using Geophysical Fluid Dynamics Laboratory's new seasonal prediction system. The locations with skillful TSI prediction show a geographic pattern that is distinct from the pattern of skillful seasonal mean SAT prediction. The prediction of TSI provides additional predictable climate information beyond the traditional seasonal mean temperature prediction. The source of the seasonal TSI prediction can be attributed to year-to-year variations of the El Niño-Southern Oscillation (ENSO), North Pacific Oscillation (NPO), and Pacific/North American (PNA) teleconnection. Over the central United States, the correlation skill of TSI prediction reaches 0.75 with strong links to observed ENSO, NPO, and PNA, while the skill of seasonal SAT prediction is relatively low with a correlation of 0.36. As a first attempt of diagnosing the combined predictability of the first-order (the seasonal mean) and second-order (TSI) statistics for SAT, this study highlights the importance of the eddy-mean flow interaction perspective for understanding the seasonal climate predictability in the extra tropics. These results point toward providing skillful prediction of higher-order statistical information related to winter temperature extremes, thus enriching the seasonal forecast products for the research community and decision makers.

Monitoring of wastewater treatment process based on multi-stage variational autoencoder

The wastewater treatment process (WWTP) is a physical and biochemical reaction process with multi-stage, non-linear and non-Gaussian characteristics. The multi-stage means that the data has diversity in different substages. In the process monitoring of the WWTP, Affinity Propagation (AP) algorithm is first used to divide the data of the WWTP into stages to process the stage characteristics of data. Next, the capability of Variational Auto-Encoder (VAE) monitoring model to restrict the Gaussian distribution in hidden layers is utilized to address the coexistence of non-Gaussian and nonlinearity. Finally, to illustrate the superiority and feasibility, the proposed model is conducted on the Benchmark Simulation Model 1 (BSM1). The experimental result illustrates that the multistage variational autoencoder (M-VAE) can extract the characteristics of data more comprehensively, with an improvement of average monitoring accuracy by 37.8%, 2.7%, 12.7% and 0.52% in 10/8 process faults compared to the state-of-the-art fault monitoring methods Auto-encoder (AE), VAE, Deep Recurrent Neural Network (DRNN), deep recurrent network with high-order statistic information (HSI-DRNN) respectively.

Fast reconstruction of multiphase microstructures based on statistical descriptors.

In this paper, we propose a hierarchical simulated annealing of erosion method (HSAE) to improve the computational efficiency of multiphase microstructure reconstruction, whose computational efficiency can be improved by an order of magnitude. Reconstruction of the two-dimensional (2D) and three-dimensional (3D) multiphase microstructures (pore, grain, and clay) based on simulated annealing (SA) and HSAE are performed. In the reconstruction of multiphase microstructure with HSAE and SA, three independent two-point correlation functions are chosen as the morphological information descriptors. The two-point cluster function which contains significant high-order statistical information is used to verify the reconstruction results. From the analysis of 2D reconstruction, it can find that the proposed HSAE technique not only improves the quality of reconstruction, but also improves the computational efficiency. The reconstructions of our proposed method are still imperfect. This is because the used two-point correlation functions contain insufficient information. For the 3D reconstruction, the two-point correlation functions of the 3D generation are in excellent agreement with those of the original 2D image, which illustrates that our proposed method is effective for the reconstruction of 3D microstructure. The comparison of the energy vs computational time between the SA and HSAE methods shows that our presented method is an order of magnitude faster than the SA method. That is because only some of the pixels in the overall hierarchy need to be considered for sampling.

Cosmology and neutrino mass with the minimum spanning tree

ABSTRACT The information content of the minimum spanning tree (MST), used to capture higher order statistics and information from the cosmic web, is compared to that of the power spectrum for a νΛCDM model. The measurements are made in redshift space using haloes from the Quijote simulation of mass $\ge 3.2\times 10^{13}\, h^{-1}\, {\rm M}_{\odot }$ in a box of length $L_{\rm box}=1\, h^{-1}\, {\rm Gpc}$. The power spectrum multipoles (monopole and quadrupole) are computed for Fourier modes in the range $0.006\, h{\rm Mpc}^{-1} \lt k \lt 0.5\, h{\rm Mpc}^{-1}$. For comparison the MST is measured with a minimum length-scale of $l_{\min }\simeq 13\, h^{-1}\, {\rm Mpc}$. Combining the MST and power spectrum allows for many of the individual degeneracies to be broken; on its own the MST provides tighter constraints on the sum of neutrino masses Mν and cosmological parameters h, ns, and Ωb but the power spectrum alone provides tighter constraints on Ωm and σ8. Combined we find constraints that are a factor of two (or greater) on all parameters with respect to the power spectrum (for Mν there is a factor of four improvement). These improvements appear to be driven by the MST’s sensitivity to small scale clustering, where the effect of neutrino free-streaming becomes relevant, and high-order statistical information in the cosmic web. The MST is shown to be a powerful tool for cosmology and neutrino mass studies, and therefore could play a pivotal role in ongoing and future galaxy redshift surveys (such as DES, DESI, Euclid, and Rubin-LSST).

Transcriptome analysis method based on differential distribution evaluation.

Identifying differential genes over conditions provides insights into the mechanisms of biological processes and disease progression. Here we present an approach, the Kullback-Leibler divergence-based differential distribution (klDD), which provides a flexible framework for quantifying changes in higher-order statistical information of genes including mean and variance/covariation. The method can well detect subtle differences in gene expression distributions in contrast to mean or variance shifts of the existing methods. In addition to effectively identifying informational genes in terms of differential distribution, klDD can be directly applied to cancer subtyping, single-cell clustering and disease early-warning detection, which were all validated by various benchmark datasets.

Digital Rock Reconstruction with User-Defined Properties Using Conditional Generative Adversarial Networks

Uncertainty is ubiquitous with multiphase flow in subsurface rocks due to their inherent heterogeneity and lack of in-situ measurements. To complete uncertainty analysis in a multi-scale manner, it is a prerequisite to provide sufficient rock samples. Even though the advent of digital rock technology offers opportunities to reproduce rocks, it still cannot be utilized to provide massive samples due to its high cost, thus leading to the development of diversified mathematical methods. Among them, two-point statistics (TPS) and multi-point statistics (MPS) are commonly utilized, which feature incorporating low-order and high-order statistical information, respectively. Recently, generative adversarial networks (GANs) are becoming increasingly popular since they can reproduce training images with excellent visual and consequent geologic realism. However, standard GANs can only incorporate information from data, while leaving no interface for user-defined properties, and thus may limit the representativeness of reconstructed samples. In this study, we propose conditional GANs for digital rock reconstruction, aiming to reproduce samples not only similar to the real training data, but also satisfying user-specified properties. In fact, the proposed framework can realize the targets of MPS and TPS simultaneously by incorporating high-order information directly from rock images with the GANs scheme, while preserving low-order counterparts through conditioning. We conduct three reconstruction experiments, and the results demonstrate that rock type, rock porosity, and correlation length can be successfully conditioned to affect the reconstructed rock images. The randomly reconstructed samples with specified rock type, porosity and correlation length will contribute to the subsequent research on pore-scale multiphase flow and uncertainty quantification.

Hierarchical-order multimodal interaction fusion network for grading gliomas

Significance. Gliomas are the most common type of primary brain tumors and have different grades. Accurate grading of a glioma is therefore significant for its clinical treatment planning and prognostic assessment with multiple-modality magnetic resonance imaging (MRI).Objective and Approach. In this study, we developed a noninvasive deep-learning method based on multimodal MRI for grading gliomas by focusing on effective multimodal fusion via leveraging collaborative and diverse high-order statistical information. Specifically, a novel high-order multimodal interaction module was designed to promote interactive learning of multimodal knowledge for more efficient fusion. For more powerful feature expression and feature correlation learning, the high-order attention mechanism is embedded in the interaction module for modeling complex and high-order statistical information to enhance the classification capability of the network. Moreover, we applied increasing orders at different levels to hierarchically recalibrate each modality stream through diverse-order attention statistics, thus encouraging all-sided attention knowledge with lesser parameters. Main results. To evaluate the effectiveness of the proposed scheme, extensive experiments were conducted on The Cancer Imaging Archive (TCIA) and Multimodal Brain Tumor Image Segmentation Benchmark 2017 (BraTS2017) datasets with five-fold cross validation to demonstrate that the proposed method can achieve high prediction performance, with area under the receiver operating characteristic curve, accuracy, sensitivity, and specificity values of 95.2%, 94.28%, 95.24%, and 92.00% on the BraTS2017 and 93.50%, 92.86%, 97.14%, and 90.48% on TCIA datasets, respectively.

An efficient ECG arrhythmia classification method based on Manta ray foraging optimization

The Electrocardiogram (ECG) arrhythmia classification has become an interesting research area for researchers and developers as it plays a vital role in early prevention and diagnosis of cardiovascular diseases. In ECG signal classification, the feature extraction and selection processes are critical steps. Thus, in this paper, different ECG signal descriptors based on one-dimensional local binary pattern (LBP), wavelet, higher-order statistical (HOS), and morphological information are introduced for feature extraction. For feature selection and classification processes, a new hybrid ECG arrhythmia classification approach called MRFO-SVM that combines a metaheuristic algorithm termed Manta ray foraging optimization (MRFO) with support vector machine (SVM) is proposed to automatically determine the relevance features of LBP, HOS, wavelet and magnitude values. In MRFO-SVM approach, the MRFO is utilized to optimize the parameters of SVM and to select the significant features subset that provides the best classification performance, meanwhile SVM is used for classification purposes. The proposed MRFO-SVM approach is trained on the MIT-BIH Arrhythmia database containing four abnormal and one normal heartbeats. The experimental results of ECG arrhythmia classification using the proposed MRFO-SVM revealed with evidence its superiority with overall classification accuracy of 98.26% over seven well-known metaheuristic algorithms.

Incremental Kernel Principal Components Subspace Inference With Nyström Approximation for Bayesian Deep Learning

As the state-of-the-art technology of Bayesian inference, based on low-dimensional principal components analysis (PCA) subspace inference methods can provide approximately accurate predictive distribution and well calibrated uncertainty. However, the main problem of PCA method is that it is a linear subspace feature extractor, and it cannot effectively represent the nonlinearly high-dimensional parameter space of deep neural networks (DNNs). Firstly, in this paper, in order to solve the main problem of the linear characteristics of PCA in high-dimensional space, we apply kernel PCA to extract higher-order statistical information in parameter space of DNNs. Secondly, to improve the efficiency of subsequent computation, we propose a strictly ordered incremental kernel PCA (InKPCA) subspace of parameter space within stochastic gradient descent (SGD) trajectories. In the proposed InKPCA subspace, we employ two approximation inference methods: elliptical slice sampling (ESS) and variational inference (VI). Finally, to further improve the memory efficiency of computing the kernel matrix, we apply Nyström approximation to determine the suitable size of subsets in the original datasets. The novelty of this paper is that it is the first time to apply the proposed InKPCA subspace with Nyström approximation for Bayesian inference in DNNs, and the results show that it can produce more accurate predictions and well-calibrated predictive uncertainty in regression and classification tasks of deep learning.

Fault diagnosis Based on An Improved Statistics Principal Analysis Similarity Factor

To settle the issue of conventional principal component analysis similarity factor cannot take advantage of higher-order statistics information of process variables, we presented an improved statistics principal analysis similarity factor (SPASF) to identify fault patterns in our work. In the improved SPASF approach, process data is first converted from original space into a new statistics space by the means of statistics pattern analysis (SPA) technology, and principal component analysis (PCA) is then employed to extract principal components in statistics space. At last, the pattern of snapshot dataset is recognized by measuring the similarity of principal components derived from statistics snapshot dataset and statistics historical fault dataset. The effectiveness of suggested SPASF based approach is verified through a case study on continuous stirring tank reactor (CSTR) by the means of recognizing fault patterns of snapshot datasets.

An effective deep recurrent network with high-order statistic information for fault monitoring in wastewater treatment process

The wastewater treatment process (WWTP) is a complex biochemical reaction process that features highly nonlinear, non-Gaussian and time correlation. As a new monitoring method, deep recurrent neural net-work (DRNN) has an effective performance in dealing with the nonlinear and time correlation of the data, but it is insufficient in dealing with the non-Gaussian characteristics. In this study, an effective deep recurrent network with high-order statistic information (HSI-DRN) is proposed for solving the insufficiency in dealing with the non-Gaussian characteristics. The proposed method extracts the high-order statistics characteristics by the over-complete independent component analysis (OICA) method. After that, weights in DRNN can be trained based on the obtained high-order statistics information and their corresponding fault labels. Because of the architecture of the network, the ability of extracting the non-Gaussian characteristics by HSI-DRN can be improved by the high-order statistics characteristics. Finally, HSI-DRN can generate visual monitoring results by discretizing the output data, which leads to more intuitively reflection of faults. Simulation studies on the BSM1 model has been performed to verify the performance of the method. For the different faults, the proposed method have higher fault monitoring ability with average false alarm rate (FAR) and missed detection rate (MAR) for respective 0.0215% and 0.586% when compared to other state-of-the-art fault monitoring methods.

Face image retrieval via sum and difference histograms of elliptical local ternary pattern

This paper presents a new feature extraction method called sum and difference histograms of elliptical local ternary pattern (SDH-ELTP) for face image retrieval. This technique first calculates sparse local ternary pattern (LTP) in an elliptical shaped neighborhood and then higher order statistical texture information is extracted via sum and difference histograms (SDH) of elliptical LTP features. In sparse elliptical LTP, a 4 point LTP from horizontal and vertical elliptical neighborhoods and 4 point LTP from simple diagonal neighborhood is considered. The SDH is calculated only in relevant directions. Since the sum and difference histogram provides higher order statistical information, the calculation of SDH of elliptical LTP features further enhances the discriminativeness of proposed descriptor. The SDH-ELTP is finally tested on two popular face image databases and the results are compared with several recent state of the art techniques. The SDH-ELTP is low dimensional and show the best retrieval results as compared to all other face image retrieval techniques.

Quality-Related Fault Detection Based on Improved Independent Component Regression for Non-Gaussian Processes

Partial least squares (PLS) and linear regression methods have been widely utilized for quality-related fault detection in industrial processes recently. Since these traditional approaches assume that process variables follow Gaussian distribution approximately, their effectiveness will be challenged when facing non-Gaussian processes. To deal with this difficulty, a new quality relevant process monitoring approach based on improved independent component regression (IICR) is presented in this article. Taking high-order statistical information into account, ICA is performed onto process data to produce independent components (ICs). In order to remove irrelevant variation orthogonal to quality variable and keep as much quality-related fault information as possible, a new quality-related independent components selection method is applied to these ICs. Then the regression relationship between filtered ICs and the product quality is built. QR decomposition for regression coefficient matrix is able to give out quality-related and quality-unrelated projectors. After the measured variable matrix is divided into quality relevant and quality irrelevant parts, novel monitoring indices are designed for fault detection. finally, applications to two simulation cases testify the effectiveness of our proposed quality-related fault detection method for non-Gaussian processes.