Complex High-dimensional Data Research Articles

Explainable machine learning approach for cancer prediction through binarilization of RNA sequencing data.

In recent years, researchers have proven the effectiveness and speediness of machine learning-based cancer diagnosis models. However, it is difficult to explain the results generated by machine learning models, especially ones that utilized complex high-dimensional data like RNA sequencing data. In this study, we propose the binarilization technique as a novel way to treat RNA sequencing data and used it to construct explainable cancer prediction models. We tested our proposed data processing technique on five different models, namely neural network, random forest, xgboost, support vector machine, and decision tree, using four cancer datasets collected from the National Cancer Institute Genomic Data Commons. Since our datasets are imbalanced, we evaluated the performance of all models using metrics designed for imbalance performance like geometric mean, Matthews correlation coefficient, F-Measure, and area under the receiver operating characteristic curve. Our approach showed comparative performance while relying on less features. Additionally, we demonstrated that data binarilization offers higher explainability by revealing how each feature affects the prediction. These results demonstrate the potential of data binarilization technique in improving the performance and explainability of RNA sequencing based cancer prediction models.

Physics-informed generator-encoder adversarial networks with latent space matching for stochastic differential equations

We propose a new class of physics-informed neural networks, called Physics-Informed Generator-Encoder Adversarial Networks, to effectively address the challenges posed by forward, inverse, and mixed problems in stochastic differential equations (SDEs). In these scenarios, while governing equations are known, the available data consist of only a limited set of snapshots for system parameters. Our model consists of two key components: the generator and the encoder, both updated alternately by gradient descent. In contrast to previous approaches that directly match approximated solutions with real snapshots, we employ an indirect matching operating within the lower-dimensional latent feature space. This method circumvents challenges associated with high-dimensional inputs and complex data distributions in solving SDEs. Numerical experiments indicate that, compared to existing deep learning solvers, our proposed approach not only demonstrates superior accuracy but also exhibits advantages in both computational efficiency and model complexity.

An overview on deep clustering

In recent years, with the great success of deep learning and especially deep unsupervised learning, many deep architectural clustering methods, collectively known as deep clustering, have emerged. Deep clustering shows the potential to outperform traditional methods, especially in handling complex high-dimensional data, taking full advantage of deep learning. To achieve a comprehensive overview of the field of deep clustering, this review systematically explores deep clustering methods and their various applications. First, the basic network architecture of deep clustering is described in detail, including the common network frameworks, and loss functions. Subsequently, deep clustering is divided into several categories based on the network architecture, and benchmark datasets and evaluation metrics in the field are introduced. Next, the real-world applications of deep clustering are explored in depth, providing successful cases in the fields of bioinformatics, medicine, anomaly detection, and image processing, highlighting the broad applicability of deep clustering in solving real-world challenges. Finally, the paper summarizes its contributions and explores potential directions for future research in deep clustering.

A deep learning‐based financial hedging approach for the effective management of commodity risks

AbstractThe development of deep learning technique has granted firms with new opportunities to substantially improve their risk management strategies for sustainable growth. This paper introduces a novel deep learning‐based financial hedging (DL‐HE) strategy to leverage the salient ability of deep learning in extracting nonlinear features from complex high dimensional data, thus boosting the management of inventory risks arising from erratic commodity prices. Using real‐world data, we find that the average annualized economic benefit of the proposed strategy is at least 1.21 million CNY for a typical aluminum firm carrying an average level of inventory in China, as compared with those of the traditional hedging strategies. Further analysis reveals that such an economic benefit can largely be explained by the efficacy of the proposed DL‐HE strategy in terms of significantly improving return while still effectively controlling risk. Moreover, the superior of this strategy remains robust when extending to copper and zinc.

Accelerated Structured Matrix Factorization

Matrix factorization exploits the idea that, in complex high-dimensional data, the actual signal typically lies in lower-dimensional structures. These lower dimensional objects provide useful insights, with interpretation favored by sparse structures. Sparsity, in addition, is beneficial in terms of regularization and, thus, to avoid over-fitting. By exploiting Bayesian shrinkage priors, we devise a computationally convenient approach for high-dimensional matrix factorization. The dependence between row and column entities is modeled by inducing flexible sparse patterns within factors. The availability of external information is accounted for in such a way that structures are allowed while not imposed. Inspired by boosting algorithms, we pair the proposed approach with a numerical strategy relying on a sequential inclusion and estimation of low-rank contributions, with a data-driven stopping rule. Practical advantages of the proposed approach are demonstrated by means of a simulation study and the analysis of soccer heatmaps obtained from new generation tracking data. Supplementary materials for this article are available online.

Multiview Deep Anomaly Detection: A Systematic Exploration.

Anomaly detection (AD), which models a given normal class and distinguishes it from the rest of abnormal classes, has been a long-standing topic with ubiquitous applications. As modern scenarios often deal with massive high-dimensional complex data spawned by multiple sources, it is natural to consider AD from the perspective of multiview deep learning. However, it has not been formally discussed by the literature and remains underexplored. Motivated by this blank, this article makes fourfold contributions: First, to the best of our knowledge, this is the first work that formally identifies and formulates the multiview deep AD problem. Second, we take recent advances in relevant areas into account and systematically devise various baseline solutions, which lays the foundation for multiview deep AD research. Third, to remedy the problem that limited benchmark datasets are available for multiview deep AD, we extensively collect the existing public data and process them into more than 30 multiview benchmark datasets via multiple means, so as to provide a better evaluation platform for multiview deep AD. Finally, by comprehensively evaluating the devised solutions on different types of multiview deep AD benchmark datasets, we conduct a thorough analysis on the effectiveness of the designed baselines and hopefully provide other researchers with beneficial guidance and insight into the new multiview deep AD topic.

Improved Selective Deep-Learning-Based Clustering Ensemble

Clustering ensemble integrates multiple base clustering results to improve the stability and robustness of the single clustering method. It consists of two principal steps: a generation step, which is about the creation of base clusterings, and a consensus function, which is the integration of all clusterings obtained in the generation step. However, most of the existing base clustering algorithms used in the generation step are shallow clustering algorithms such as k-means. These shallow clustering algorithms do not work well or even fail when dealing with large-scale, high-dimensional unstructured data. The emergence of deep clustering algorithms provides a solution to address this challenge. Deep clustering combines the unsupervised commonality of deep representation learning to address complex high-dimensional data clustering, which has achieved excellent performance in many fields. In light of this, we introduce deep clustering into clustering ensemble and propose an improved selective deep-learning-based clustering ensemble algorithm (ISDCE). ISDCE exploits the deep clustering algorithm with different initialization parameters to generate multiple diverse base clusterings. Next, ISDCE constructs ensemble quality and diversity evaluation metrics of base clusterings to select higher-quality and rich-diversity candidate base clusterings. Finally, a weighted graph partition consensus function is utilized to aggregate the candidate base clusterings to obtain a consensus clustering result. Extensive experimental results on various types of datasets demonstrate that ISDCE performs significantly better than existing clustering ensemble approaches.

Mathematical scheduling model of complex industrial process combining swarm intelligence algorithm and swarm dimension reduction technology

The swarm intelligence algorithm is widely used in scheduling problems due to its superior searchability and adaptability, but it faces high computational costs and low efficiency when dealing with high-dimensional complex data. To enhance the handling of large-scale and intricate data, as well as enhance the accuracy and efficiency of scheduling, this study develops a mathematical model for industrial scheduling. The model is based on swarm intelligence algorithms such as symbiotic evolution algorithms, grey wolf algorithms, cuckoo algorithms, population dimensionality reduction techniques, and resource optimal allocation calculation methods. The final test results showed that the accuracy of the model in calculating the objective function was about 80 % in cases of high complexity and about 90 % in cases of low complexity. Compared with other algorithms, its accuracy was improved by 7 %, error rate was reduced by 50 %, time cost was reduced by about 30 %, and conversion rate was fast and more stable on average after 200 generations. The experimental results indicate that the proposed mathematical scheduling model, which integrates multiple intelligent algorithms and population dimensionality reduction technologies, has certain practices and feasibility in large-scale and multi-dimensional production for enterprises. This study combines swarm intelligence algorithms with population dimensionality reduction techniques, which is an innovative attempt at complex industrial scheduling problems.

Risk of papillary thyroid carcinoma and nodular goiter associated with exposure to semi-volatile organic compounds: A multi-pollutant assessment based on machine learning algorithms

BackgroundExposure to semi-volatile organic compounds (SVOCs) may link to thyroid nodule risk, but studies of mixed-SVOCs exposure effects are lacking. Traditional analytical methods are inadequate for dealing with mixed exposures, while machine learning (ML) seems to be a good way to fill the gaps in the field of environmental epidemiology research. ObjectivesDifferent ML algorithms were used to explore the relationship between mixed-SVOCs exposure and thyroid nodule. MethodsA 1:1:1 age- and gender-matched case-control study was conducted in which 96 serum SVOCs were measured in 50 papillary thyroid carcinoma (PTC), 50 nodular goiters (NG), and 50 controls. Different ML techniques such as Random Forest, AdaBoost were selected based on their predictive power, and variables were selected based on their weights in the models. Weighted quantile sum (WQS) regression and Bayesian kernel machine regression (BKMR) were used to assess the mixed effects of the SVOCs exposure on thyroid nodule. ResultsForty-three of 96 SVOCs with detection rate >80 % were included in the analysis. ML algorithms showed a consistent selection of SVOCs associated with thyroid nodule. Fluazifop-butyl and fenpropathrin are positively associated with PTC and NG in single compound models (all P < 0.05). WQS model shows that exposure to mixed-SVOCs was associated with an increased risk of PTC and NG, with the mixture dominated by fenpropathrin, followed by fluazifop-butyl and propham. In the BKMR model, mixtures showed a significant positive association with thyroid nodule risk at high exposure levels, and fluazifop-butyl showed positive effects associated with PTC and NG. ConclusionThis study confirms the feasibility of ML methods for variable selection in high-dimensional complex data and showed that mixed exposure to SVOCs was associated with increased risk of PTC and NG. The observed association was primarily driven by fluazifop-butyl and fenpropathrin. The findings warranted further investigation.

Synchrotron radiation data-driven artificial intelligence approaches in materials discovery

Synchrotron radiation technology provides high-resolution and high-sensitivity information for many fields such as material science, life science, and energy research. Synchrotron radiation data-driven methods have significantly accelerated the development of materials discovery and analysis. However, synchrotron radiation data is complex and large, requiring artificial intelligence for analysis. Artificial intelligence can efficiently process complex high-dimensional data, automate the analysis process, discover hidden patterns and associations, and build predictive models. This review provides an overview of the application and development of combining synchrotron radiation data-driven methods with artificial intelligence in the field of materials discovery. The application of the method in science is still limited by the problems of large and complex synchrotron radiation data, valuable experimental machine time, and uninterpretable artificial intelligence models. To address these problems, this review correspondingly proposes solutions for synchrotron radiation artificial intelligence data banks, standardized experiment records systems, and interpretable artificial intelligence predictive models.

A Synergistic MOEA Algorithm with GANs for Complex Data Analysis

The multi-objective evolutionary algorithm optimization (MOEA) is a challenging but critical approach for tackling complex data analysis problems. However, prevailing MOEAs often rely on single strategies to obtain optimal solutions, leading to concerns such as premature convergence and insufficient population diversity, particularly in high-dimensional data scenarios. In this paper, we propose a novel adversarial population generation algorithm, APG-SMOEA, which synergistically combines the benefits of MOEAs and Generative Adversarial Networks (GANs) to address these limitations. In order to balance the efficiency and quality of offspring selection, we introduce an adaptive population entropy strategy, which includes control parameters based on population entropy and a learning pool for storing and retrieving optimal solutions. Additionally, we attempt to alleviate the training complexity and model collapse problems common in GANs with APG-SMOEA. Experimental results on benchmarks demonstrate that the proposed algorithm is superior to the existing algorithms in terms of solution quality and diversity of low-dimensional or high-dimensional complex data.

Feature Extraction and Classification Analysis of High-Dimensional Biological Data Based on Dimensionality Reduction Fusion Method

Identification and extraction of characterized information from complex high-dimensional biological data is a very meaningful issue. The dimensionality reduction fusion method based on random forest, feature extraction and neural network is proposed to recognize and classify two datasets of mRNA and lncRNA. It is shown that the proposed fusion method achieved accurate identification/classification of cancer and non-cancer groups, and simultaneously selected identity variables that have biological relevance to lung cancer (tumor) as potential biomarkers from a large number of variables. It is considered as an effective tool and theoretical support for lung cancer identification in clinical application, and it can be extended to other kinds of cancer or biological data. Ultimately, an advanced method for feature extraction and classification analysis of high-dimensional data is provided.

Repeated Decision Stumping Distils Simple Rules from Single-Cell Data.

Single-cell data afford unprecedented insights into molecular processes. But the complexity and size of these data sets have proved challenging and given rise to a large armory of statistical and machine learning approaches. The majority of approaches focuses on either describing features of these data, or making predictions and classifying unlabeled samples. In this study, we introduce repeated decision stumping (ReDX) as a method to distill simple models from single-cell data. We develop decision trees of depth one-hence "stumps"-to identify in an inductive manner, gene products involved in driving cell fate transitions, and in applications to published data we are able to discover the key players involved in these processes in an unbiased manner without prior knowledge. Our algorithm is deliberately targeting the simplest possible candidate hypotheses that can be extracted from complex high-dimensional data. There are three reasons for this: (1) the predictions become straightforwardly testable hypotheses; (2) the identified candidates form the basis for further mechanistic model development, for example, for engineering and synthetic biology interventions; and (3) this approach complements existing descriptive modeling approaches and frameworks. The approach is computationally efficient, has remarkable predictive power, including in simulation studies where the ground truth is known, and yields robust and statistically stable predictors; the same set of candidates is generated by applying the algorithm to different subsamples of experimental data.

Support Matrix Machine via Joint l2,1 and Nuclear Norm Minimization Under Matrix Completion Framework for Classification of Corrupted Data.

Traditional support vector machines (SVMs) are fragile in the presence of outliers; even a single corrupt data point can arbitrarily alter the quality of the approximation. If even a small fraction of columns is corrupted, then classification performance will inevitably deteriorate. This article considers the problem of high-dimensional data classification, where a number of the columns are arbitrarily corrupted. An efficient Support Matrix Machine that simultaneously performs matrix Recovery (SSMRe) is proposed, i.e. feature selection and classification through joint minimization of l2,1 (the nuclear norm of L ). The data are assumed to consist of a low-rank clean matrix plus a sparse noisy matrix. SSMRe works under incoherence and ambiguity conditions and is able to recover an intrinsic matrix of higher rank in the presence of data densely corrupted. The objective function is a spectral extension of the conventional elastic net; it combines the property of matrix recovery along with low rank and joint sparsity to deal with complex high-dimensional noisy data. Furthermore, SSMRe leverages structural information, as well as the intrinsic structure of data, avoiding the inevitable upper bound. Experimental results on different real-time applications, supported by the theoretical analysis and statistical testing, show significant gain for BCI, face recognition, and person identification datasets, especially in the presence of outliers, while preserving a reasonable number of support vectors.

A Neural Networks Based Model to Predict the Interest of College Students in Sports Activities

The widespread application of big data technology in various fields, including research in education and sports in colleges, has also been deeply influenced. College students are the future strength of a country, and their habits and interests in sports activities have profound significance for their physical and mental health, teamwork, and outlook on life. However, traditional research methods, such as questionnaire surveys, observations, or interviews, have obvious limitations when dealing with large amounts of complex high-dimensional data. This study aimed to extract the interesting features of college students regarding sports activities using graph neural network (GNN) technology. Then, the labels of those interest features were further predicted, and a feature matrix was constructed. Finally, the K-means clustering method was used to achieve accurate feature clustering. This study presents a novel idea and approach for physical education and event planning in colleges, offering both practical value and theoretical significance.

A lightweight IoT intrusion detection model based on improved BERT-of-Theseus

The proliferation of Internet of Things (IoT) technology has resulted in an increase in security vulnerabilities associated with the interconnectivity of IoT devices. As a result, there is a need for intrusion detection mechanisms that can effectively detect attacks on IoT security vulnerabilities. However, due to the resource constraints of IoT deployment devices, intrusion detection schemes must be customized to meet the specific demands of the IoT environment. In this study, we propose a knowledge-distillation-based IoT intrusion detection model named BT-TPF, which is capable of detecting network attacks encountered by IoT devices in an IoT environment with limited computing resources. The proposed BT-TPF model leverages a Siamese network for feature dimensionality reduction of complex high-dimensional network traffic data. Additionally, it employs a large-scale Vision Transformer as a teacher model to guide a small-scale Poolformer model during training, before deploying the trained Poolformer model as a classifier to detect network intrusion traffic. Through knowledge distillation, the final small model obtained in this paper only requires a minimum of 788 parameters, reducing the number of parameters by approximately 90% compared to the large model before knowledge distillation, while maintaining high detection accuracy. Experimental results show that the BT-TPF model achieves over 99% accuracy on both the CIC-IDS2017 and TON_IoT datasets. Furthermore, it exhibits significant advantages compared to traditional Deep Learning methods and recent state-of-the-art models, as evidenced by various evaluation metrics.

Anomaly Detection for Wind Turbines Using Long Short-Term Memory-Based Variational Autoencoder Wasserstein Generation Adversarial Network under Semi-Supervised Training

Intelligent anomaly detection for wind turbines using deep-learning methods has been extensively researched and yielded significant results. However, supervised learning necessitates sufficient labeled data to establish the discriminant boundary, while unsupervised learning lacks prior knowledge and heavily relies on assumptions about the distribution of anomalies. A long short-term memory-based variational autoencoder Wasserstein generation adversarial network (LSTM-based VAE-WGAN) was established in this paper to address the challenge of small and noisy wind turbine datasets. The VAE was utilized as the generator, with LSTM units replacing hidden layer neurons to effectively extract spatiotemporal factors. The similarity between the model-fit distribution and true distribution was quantified using Wasserstein distance, enabling complex high-dimensional data distributions to be learned. To enhance the performance and robustness of the proposed model, a two-stage adversarial semi-supervised training approach was implemented. Subsequently, a monitoring indicator based on reconstruction error was defined, with the threshold set at a 99.7% confidence interval for the distribution curve fitted by kernel density estimation (KDE). Real cases from a wind farm in northeast China have confirmed the feasibility and advancement of the proposed model, while also discussing the effects of various applied parameters.

Hypergraph-Clustering Method Based on an Improved Apriori Algorithm

With the complexity and variability of data structures and dimensions, traditional clustering algorithms face various challenges. The integration of network science and clustering has become a popular field of exploration. One of the main challenges is how to handle large-scale and complex high-dimensional data effectively. Hypergraphs can accurately represent multidimensional heterogeneous data, making them important for improving clustering performance. In this paper, we propose a hypergraph-clustering method dubbed the “high-dimensional data clustering method” based on hypergraph partitioning using an improved Apriori algorithm (HDHPA). First, the method constructs a hypergraph based on the improved Apriori association rule algorithm, where frequent itemsets existing in high-dimensional data are treated as hyperedges. Then, different frequent itemsets are mined in parallel to obtain hyperedges with corresponding ranks, avoiding the generation of redundant rules and improving mining efficiency. Next, we use the dense subgraph partition (DSP) algorithm to divide the hypergraph into multiple subclusters. Finally, we merge the subclusters through dense sub-hypergraphs to obtain the clustering results. The advantage of this method lies in its use of the hypergraph model to discretize the association between data in space, which further enhances the effectiveness and accuracy of clustering. We comprehensively compare the proposed HDHPA method with several advanced hypergraph-clustering methods using seven different types of high-dimensional datasets and then compare their running times. The results show that the clustering evaluation index values of the HDHPA method are generally superior to all other methods. The maximum ARI value can reach 0.834, an increase of 42%, and the average running time is lower than other methods. All in all, HDHPA exhibits an excellent comparable performance on multiple real networks. The research results of this paper provide an effective solution for processing and analyzing large-scale network datasets and are also conducive to broadening the application range of clustering techniques.

Takagi-Sugeno-Kang fuzzy system fusion: A survey at hierarchical, wide and stacked levels

With excellent global approximation performance and interpretability, Takagi-Sugeno-Kang (TSK) fuzzy systems have enjoyed a wide range of applications in various fields, such as smart control, medical, and finance. However, in handling high-dimensional complex data, the performance and interpretability of a single TSK fuzzy system are easily degraded by rule explosion due to the curse of dimensionality. Ensemble learning comes into play to deal with the problem by the fusion of multiple TSK fuzzy systems using appropriate ensemble learning strategies, which has shown to be effective in eliminating the issue of the curse of dimensionality curse problem and reducing the number of fuzzy rules, thereby maintaining the interpretability of fuzzy systems. To this end, this paper gives a comprehensive survey of TSK fuzzy system fusion to provide insights into further research development. First, we briefly review the fundamental concepts related to TSK fuzzy systems, including fuzzy rule structures, training methods, and interpretability, and discuss the three different development directions of TSK fuzzy systems. Next, along the direction of TSK fuzzy system fusion, we investigate in detail the current ensemble strategies for fusion at hierarchical, wide and stacked levels, and discuss their differences, merits and weaknesses from the aspects of time complexity, interpretability (model complexity) and classification performance. We then present some applications of TSK fuzzy systems in real-world scenarios. Finally, the challenges and future directions of TSK fuzzy system fusion are discussed to foster prospective research.

Drought prediction: Insights from the fusion of LSTM and multi-source factors

In the context of current global climate change, accurate drought prediction is crucial for water resources management and agricultural production. Although traditional drought forecasting methods largely rely on historical climatic data, these methods cannot fully consider the long-term effects of factors, such as climate change, and the evaluation of prediction results is limited. Therefore, this study proposed a drought prediction and evaluation framework based on Long Short-Term Memory (LSTM), integrating multi-source factors to significantly enhance the accuracy and reliability of drought prediction models. This framework applied two distinct forecasting schemes. The first scheme utilized ten diverse factors, including precipitation, evaporation, bare soil percentage area coverage, percentage crop cover, leaf area index, runoff, surface runoff, soil moisture, temperature, and total vegetated percentage cover, to predict future precipitation and evaporation, which was then used to calculate the Standardized Precipitation-Evaporation Index (SPEI) to evaluate drought characteristics. The second scheme directly used these ten factors and historical SPEI to predict future SPEI, further assessing future drought characteristics. By comparing the drought prediction results of the two schemes in terms of data statistics, drought characteristics, and spatial patterns, it was found that the LSTM model significantly improved accuracy when handling high-dimensional complex data and predicting key factors such as precipitation, evaporation, temperature, and soil moisture. The first scheme was more accurate when predicting severe and extreme droughts, whereas the second scheme was more sensitive to predicting moderate and mild droughts and exhibited higher stability and regularity in predicting the spatial variability of drought. In summary, LSTM has made significant improvements in the accuracy, stability, and reliability of drought prediction, providing stronger support for practical applications, such as agriculture and water resources management, and offering a new research tool for further climate change research.