High-dimensional Datasets Research Articles

Predicting financial distress in high-dimensional imbalanced datasets: a multi-heterogeneous self-paced ensemble learning framework

Financial distress prediction (FDP) is a critical area of study for researchers, industry stakeholders, and regulatory authorities. However, FDP tasks present several challenges, including high-dimensional datasets, class imbalances, and the complexity of parameter optimization. These issues often hinder the predictive model’s ability to accurately identify companies at high risk of financial distress. To mitigate these challenges, we introduce FinMHSPE—a novel multi-heterogeneous self-paced ensemble (MHSPE) FDP learning framework. The proposed model uses pairwise comparisons of data from multiple time frames combined with the maximum relevance and minimum redundancy method to select an optimal subset of features, effectively resolving the high dimensionality issue. Furthermore, the proposed framework incorporates the MHSPE model to iteratively identify the most informative majority class data samples, effectively addressing the class imbalance issue. To optimize the model’s parameters, we leverage the particle swarm optimization algorithm. The robustness of our proposed model is validated through extensive experiments performed on a financial dataset of Chinese listed companies. The empirical results demonstrate that the proposed model outperforms existing competing models in the field of FDP. Specifically, our FinMHSPE framework achieves the highest performance, achieving an area under the curve (AUC) value of 0.9574, considerably surpassing all existing methods. A comparative analysis of AUC values further reveals that FinMHSPE outperforms state-of-the-art approaches that rely on financial features as inputs. Furthermore, our investigation identifies several valuable features for enhancing FDP model performance, notably those associated with a company’s information and growth potential.

Efficient Distributed Sparse Relative Similarity Learning

Learning a good similarity measure for large-scale high-dimensional data is a crucial task in machine learning applications, yet it poses a significant challenge. Distributed minibatch stochastic gradient descent (SGD) serves as an efficient optimization method in large-scale distributed training, allowing linear speedup in proportion to the number of workers. However, communication efficiency in distributed SGD requires a sufficiently large minibatch size, presenting two distinct challenges. Firstly, a large minibatch size leads to high memory usage and computational complexity during parallel training of high-dimensional models. Second, a larger batch size of data reduces the convergence rate. To overcome these challenges, we propose an efficient distributed sparse relative similarity learning framework EDSRSL . This framework integrates two strategies: local minibatch SGD and sparse relative similarity learning. By effectively reducing the number of updates through synchronous delay while maintaining a large batch size, we address the issue of high computational cost. Additionally, we incorporate sparse model learning into the training process, significantly reducing computational cost. This paper also provides theoretical proof that the convergence rate does not decrease significantly with increasing batch size. Various experiments on six high-dimensional real-world datasets demonstrate the efficacy and efficiency of the proposed algorithms, with a communication cost reduction of up to \(90.89\%\) and a maximum wall time speedup of \(5.66\times\) compared to the baseline methods.

Differentially private low-dimensional synthetic data from high-dimensional datasets

Differentially private low-dimensional synthetic data from high-dimensional datasets

A Randomized Response Framework to Achieve Differential Privacy in Medical Data

In recent years, differential privacy has gained substantial traction in the medical domain, where the need to balance privacy preservation with data utility is paramount. As medical data increasingly relies on cloud platforms and distributed sharing among multiple stakeholders, such as healthcare providers, researchers, and policymakers, the importance of privacy-preserving techniques has become more pronounced. Trends in the field focus on designing efficient algorithms tailored to high-dimensional medical datasets, incorporating privacy guarantees into federated learning for distributed medical devices, and addressing challenges posed by adversarial attacks. Our work lays a foundation for these emerging applications by emphasizing the role of randomized response within the broader differential privacy framework, paving the way for advancements in secure medical data sharing and analysis. In this paper, we analyze the classical concept of a randomized response and investigate how it relates to the fundamental concept of differential privacy. Our approach is both mathematical and algorithmic in nature, and our purpose is twofold. On the one hand, we provide a formal and precise definition of differential privacy within a natural and convenient probabilistic—statistical framework. On the other hand, we position a randomized response as a special yet significant instance of differential privacy, demonstrating its utility in preserving individual privacy in sensitive data scenarios. To substantiate our findings, we include key theoretical proofs and provide indicative simulations, accompanied by open-access code to facilitate reproducibility and further exploration.

SymbolNet: Neural symbolic regression with adaptive dynamic pruning for compression

Abstract Compact symbolic expressions have been shown to be more efficient than neural network models in terms of resource consumption and inference speed when implemented on custom hardware such as FPGAs, while maintaining comparable accuracy.
These capabilities are highly valuable in environments with stringent computational resource constraints, such as high-energy physics experiments at the CERN Large Hadron Collider.
However, finding compact expressions for high-dimensional datasets remains challenging due to the inherent limitations of genetic programming, the search algorithm of most symbolic regression methods.
Contrary to genetic programming, the neural network approach to symbolic regression offers scalability to high-dimensional inputs and leverages gradient methods for faster equation searching.
Common ways of constraining expression complexity often involve multistage pruning with fine-tuning, which can result in significant performance loss.
In this work, we propose $\tt{SymbolNet}$, a neural network approach to symbolic regression specifically designed as a model compression technique, aimed at enabling low-latency inference for high-dimensional inputs on custom hardware such as FPGAs.
This framework allows dynamic pruning of model weights, input features, and mathematical operators in a single training process, where both training loss and expression complexity are optimized simultaneously.
We introduce a sparsity regularization term for each pruning type, which can adaptively adjust its strength, leading to convergence at a target sparsity ratio.
Unlike most existing symbolic regression methods that struggle with datasets containing more than $\mathcal{O}(10)$ inputs, we demonstrate the effectiveness of our model on the LHC jet tagging task (16 inputs), MNIST (784 inputs), and SVHN (3072 inputs).

Machine Learning Methods for Weather Forecasting: A Survey

Weather forecasting, a vital task for agriculture, transportation, energy, etc., has evolved significantly over the years. Comprehensive surveys play a crucial role in synthesizing knowledge, identifying trends, and addressing emerging challenges in this dynamic field. In this survey, we critically examines machine learning (ML)-based weather forecasting methods, which demonstrate exceptional capability in handling complex, high-dimensional datasets and leveraging large volumes of historical and real-time data, enabling the identification of subtle patterns and relationships among weather variables. Research on specific tasks such as global weather forecasting, downscaling, extreme weather prediction, and how to combine machine learning methods with physical principles are very active in the current field. However, several unresolved or challenging issues remain, including the interpretability of models and the ability to predict rare weather events. By identifying these gaps, this research provides a roadmap for advancing machine learning-based weather forecasting techniques to complement and enhance weather prediction results.

A Selective Under-Sampling (SUS) Method for Imbalanced Regression

Many mainstream machine learning approaches, such as neural networks, are not well suited to work with imbalanced data. Yet, this problem is frequently present in many real-world data sets. Collection methods are imperfect, and often not able to capture enough data in a specific range of the target variable. Furthermore, in certain tasks data is inherently imbalanced with many more normal events than edge cases. This problem is well studied within the classification context. However, only several methods have been proposed to deal with regression tasks. In addition, the proposed methods often do not yield good performance with high-dimensional data, while imbalanced high-dimensional regression has scarcely been explored. In this paper we present a selective under-sampling (SUS) algorithm for dealing with imbalanced regression and its iterative version SUSiter. We assessed this method on 15 regression data sets from different imbalanced domains, 5 synthetic high-dimensional imbalanced data sets and 2 more complex imbalanced age estimation image data sets. Our results suggest that SUS and SUSiter typically outperform other state-of-the-art techniques like SMOGN, or random under-sampling, when used with neural networks as learners.

Distance-based mutual congestion feature selection with genetic algorithm for high-dimensional medical datasets

Distance-based mutual congestion feature selection with genetic algorithm for high-dimensional medical datasets

Leveraging neural networks for robust survival estimation in cox proportional hazards model with censored data

Survival analysis, traditionally dominated by models like the Cox Proportional Hazards, faces significant challenges in handling high-dimensional and complex datasets common in modern medical and biological research. This paper introduces an innovative approach to address these challenges by leveraging Variational Autoencoders (VAE) for feature extraction before survival analysis. We propose a framework where VAE compress high-dimensional input data into a lower-dimensional, yet informative latent space. Also, we identify key factors associated with mortality. We use publicly available Covid-19 data and analyze the survival characteristics of a large sample of 566,602 patients. Various machine learning techniques compared to improve prediction accuracy. The performance of the models has been evaluated using metrics like the concordance index and accuracy. The proposed algorithm performs the best with an accuracy of 0.94. Also, it shows good concordance, with training and test concordance index values of 0.91559 and 0.91299, respectively, while other algorithms range from 70% to 91% accuracy. The findings suggest that most features have a significant impact on reducing mortality, except for COPD, asthma, inmsupr, cardiovascular, obesity, renal chronic and tobacco. This model can help identify patients at high risk of death, so they can be prioritized for critical care.

Unsupervised self-organising map classification of Raman spectra from prostate cell lines uncovers substratified prostate cancer disease states

Prostate cancer is a disease which poses an interesting clinical question: Should it be treated? Only a small subset of prostate cancers are aggressive and require removal and treatment to prevent metastatic spread. However, conventional diagnostics remain challenged to risk-stratify such patients; hence, new methods of approach to biomolecularly sub-classify the disease are needed. Here we use an unsupervised self-organising map approach to analyse live-cell Raman spectroscopy data obtained from prostate cell-lines; our aim is to exemplify this method to sub-stratify, at the single-cell-level, the cancer disease state using high-dimensional datasets with minimal preprocessing. The results demonstrate a new sub-clustering of the prostate cancer cell-line into two groups—protein-rich and lipid-rich sub-cellular components—which we believe to be mechanistically linked. This finding shows the potential for unsupervised machine learning to discover distinct disease-state features for more accurate characterisation of highly heterogeneous prostate cancer. Applications may lead to more targeted diagnoses, prognoses and clinical treatment decisions via molecularly-informed stratification that would benefit patients. A method that could discover distinct disease-state features that are mechanistically linked could also assist in the development of more effective broad-spectrum treatments that simultaneously target linked disease-state processes.

Beyond Correlation: Incorporating Counterfactual Guidance to Better Support Exploratory Visual Analysis.

Providing effective guidance for users has long been an important and challenging task for efficient exploratory visual analytics, especially when selecting variables for visualization in high-dimensional datasets. Correlation is the most widely applied metric for guidance in statistical and analytical tools, however a reliance on correlation may lead users towards false positives when interpreting causal relations in the data. In this work, inspired by prior insights on the benefits of counterfactual visualization in supporting visual causal inference, we propose a novel, simple, and efficient counterfactual guidance method to enhance causal inference performance in guided exploratory analytics based on insights and concerns gathered from expert interviews. Our technique aims to capitalize on the benefits of counterfactual approaches while reducing their complexity for users. We integrated counterfactual guidance into an exploratory visual analytics system, and using a synthetically generated ground-truth causal dataset, conducted a comparative user study and evaluated to what extent counterfactual guidance can help lead users to more precise visual causal inferences. The results suggest that counterfactual guidance improved visual causal inference performance, and also led to different exploratory behaviors compared to correlation-based guidance. Based on these findings, we offer future directions and challenges for incorporating counterfactual guidance to better support exploratory visual analytics.

Improving Generalization of Genetic Programming for High-Dimensional Symbolic Regression with Shapley Value Based Feature Selection

AbstractSymbolic Regression (SR) on high-dimensional datasets often encounters significant challenges, resulting in models with poor generalization capabilities. While feature selection has the potential to enhance the generalization and learning performance in general, its application in Genetic Programming (GP) for high-dimensional SR remains a complex problem. Originating from game theory, the Shapley value is applied to additive feature attribution approaches where it distributes the difference between a model output and a baseline average across input variables. By providing an accurate assessment of each feature importance, the Shapley value offers a robust approach to select features. In this paper, we propose a novel feature selection method leveraging the Shapley value to identify and select important features in GP for high-dimensional SR. Through a series of experiments conducted on ten high-dimensional regression datasets, the results indicate that our algorithm surpasses standard GP and other GP-based feature selection methods in terms of learning and generalization performance on most datasets. Further analysis reveals that our algorithm generates more compact models, focusing on the inclusion of important features.

Hybrid rice optimization algorithm inspired grey wolf optimizer for high-dimensional feature selection

Feature selection (FS) is a significant dimensionality reduction technique, which can effectively remove redundant features. Metaheuristic algorithms have been widely employed in FS, and have obtained satisfactory performance, among them, grey wolf optimizer (GWO) has received widespread attention. However, the GWO and its variants suffer from limited adaptability, poor diversity, and low accuracy when faced with high-dimensional data. The hybrid rice optimization (HRO) algorithm is an emerging metaheuristic algorithm derived from the hybrid heterosis and breeding mechanism in nature. It possesses a robust capacity to identify and converge towards optimal solutions. Therefore, a novel approach based on multi-strategy collaborative GWO combined with the HRO algorithm (HRO-GWO) for FS is proposed in this paper. The HRO-GWO algorithm is enhanced by four innovative strategies including dynamical regulation strategy and three search strategies. First, to improve the adaptability of GWO, the dynamical regulation strategy is devised for parameter optimization of GWO. Then, a multi-strategy co-evolution model inspired by HRO is designed, which utilizes neighborhood search, dual-crossover, and selfing techniques to bolster population diversity. Finally, the study develops a hybrid filter-wrapper framework incorporating chi-square and the HRO-GWO algorithm to efficiently select pertinent and informative feature subsets, enhancing the classification performance while conserving time. The performance of HRO-GWO has been rigorously assessed across benchmark functions and the effectiveness of the proposed framework has been evaluated on small-sample high-dimensional biomedical datasets. Our experimental findings demonstrate that the approach on the basis of HRO-GWO outperforms state−of-the−art methods.

An efficient method for identifying surface damage in hydraulic concrete buildings

Traditional hydraulic structures rely on manual visual inspection for apparent integrity, which is not only time-consuming and labour-intensive but also inefficient. The efficacy of deep learning models is frequently constrained by the size of available data, resulting in limited scalability and flexibility. Furthermore, the paucity of data diversity leads to a singular function of the model that cannot provide comprehensive decision support for improving maintenance measures. This paper proposes an efficacious methodology for identifying diverse apparent damages in hydraulic structures to address the limitations of existing technologies. The advanced features of apparent damage in hydraulic structures were elucidated by fine-tuning the top-level parameters of the lightweight pre-trained model, thereby mitigating the data dependency issue inherent in the model. Ensemble learning algorithms are employed to classify high-dimensional samples to enhance the accuracy and stability of the classification. However, ensemble learning algorithms are subject to time consuming issues when applied to high-dimensional datasets. To this end, we propose a robust discriminative feature selection model to identify the most salient features, thereby enhancing the performance of apparent damage recognition in hydraulic structures while concurrently reducing the inference time. The results demonstrated that the accuracies of this method in identifying crack, fracture, hole and normal structures were 87.65%, 87.82%, 96.99%, and 95.25%, respectively. This methodology exhibits significant applicability and practical value for the intelligent inspection of hydraulic structures.

An effective heuristic for developing hybrid feature selection in high dimensional and low sample size datasets.

High-dimensional datasets with low sample sizes (HDLSS) are pivotal in the fields of biology and bioinformatics. One of core objective of HDLSS is to select most informative features and discarding redundant or irrelevant features. This is particularly crucial in bioinformatics, where accurate feature (gene) selection can lead to breakthroughs in drug development and provide insights into disease diagnostics. Despite its importance, identifying optimal features is still a significant challenge in HDLSS. To address this challenge, we propose an effective feature selection method that combines gradual permutation filtering with a heuristic tribrid search strategy, specifically tailored for HDLSS contexts. The proposed method considers inter-feature interactions and leverages feature rankings during the search process. In addition, a new performance metric for the HDLSS that evaluates both the number and quality of selected features is suggested. Through the comparison of the benchmark dataset with existing methods, the proposed method reduced the average number of selected features from 37.8 to 5.5 and improved the performance of the prediction model, based on the selected features, from 0.855 to 0.927. The proposed method effectively selects a small number of important features and achieves high prediction performance.

GDSMOTE: A Novel Synthetic Oversampling Method for High-Dimensional Imbalanced Financial Data

Synthetic oversampling methods for dealing with imbalanced classification problems have been widely studied. However, the current synthetic oversampling methods still cannot perform well when facing high-dimensional imbalanced financial data. The failure of distance measurement in high-dimensional space, error accumulation caused by noise samples, and the reduction of recognition accuracy of majority samples caused by the distribution of synthetic samples are the main reasons that limit the performance of current methods. Taking these factors into consideration, a novel synthetic oversampling method is proposed, namely the gradient distribution-based synthetic minority oversampling technique (GDSMOTE). Firstly, the concept of gradient contribution was used to assign the minority-class samples to different gradient intervals instead of relying on the spatial distance. Secondly, the root sample selection strategy of GDSMOTE avoids the error accumulation caused by noise samples and a new concept of nearest neighbor was proposed to determine the auxiliary samples. Finally, a safety gradient distribution approximation strategy based on cosine similarity was designed to determine the number of samples to be synthesized in each safety gradient interval. Experiments on high-dimensional imbalanced financial datasets show that GDSMOTE can achieve a higher F1-Score and MCC metrics than baseline methods while achieving a higher recall score. This means that our method has the characteristics of improving the recognition accuracy of minority-class samples without sacrificing the recognition accuracy of majority-class samples and has good adaptability to data decision-making tasks in the financial field.

An Integrated Algorithm with Feature Selection, Data Augmentation, and XGBoost for Ovarian Cancer

Ovarian cancer is one of the most aggressive gynecological cancers due to its high invasion and chemoresistance. It not only has a high incidence rate but also tops the list of mortality rates. Its subtle early symptoms make subsequent diagnosis difficult, significantly delaying timely treatment for patients. Once ovarian cancer reaches an advanced stage, the complexity and difficulty of treatment increase substantially, affecting patient survival rates. Therefore, it is crucial for both medical professionals and patients to remain highly vigilant about the early signs of ovarian cancer to ensure timely intervention. In recent years, ovarian cancer prediction research has advanced, allowing for the analysis of the likelihood and type of cancer based on patients’ genetic data. With the rapid development of machine learning, numerous efficient classification prediction models have emerged. These new technologies offer significant opportunities and potential for developing ovarian cancer diagnostic prediction methods. However, traditional approaches often struggle to achieve satisfactory classification accuracy in high-dimensional genetic datasets with small sample sizes. This research offers a prediction model utilizing genomic data to enhance the early diagnosis rate of ovarian cancer, incorporating feature selection, data augmentation through adversarial conditional generative adversarial networks (AC-GAN), and an extreme gradient boosting (XGBoost) classifier. First, we can simplify the original genetic dataset through feature selection methods, removing irrelevant variables and noise, thereby improving the model’s predictive accuracy. Following dimensionality reduction, AC-GAN enriches the data, producing more realistic genetic samples to enhance the model’s generalization capacity. Finally, the XGBoost classifier is applied to classify the augmented data, achieving efficient predictions for ovarian cancer. These research findings strongly demonstrate that the diagnostic method proposed in this paper has a significant advantage in the predictive diagnosis of ovarian cancer, with an accuracy of 99.01% that surpasses the current technologies in use. Additionally, the algorithm identifies twelve genes highly relevant to ovarian cancer, providing valuable insights for physicians during diagnosis.

Comparison of Feature Extraction Methods in High Dimensional Time Series

Working with high-dimensional datasets increases the workload on machine learning models. Therefore, before making predictions, the most meaningful data points in the entire data set must be determined. It is highly important to improve model performance, especially in the field of machine learning. For this reason, five feature selection methods—Mutual Information, Principal Component Analysis, Chi-square, Information Gain, and Variance Thresholding—commonly used in the literature, were tested on the 14400 feature data set obtained with a system previously proposed to determine the sand, silt and clay ratios in the soil. The success of these five methods is presented comparatively using R-square (R²) and Mean Absolute Error (MAE) metrics. The best results were obtained with the Information Gain method for sand (R2 = 0.44), with Chi-square for silt (R2 = 0.17), and with Variance Thresholding for clay (R2 = 0.61).

Fast spatial simulation of extreme high-resolution radar precipitation data using integrated nested Laplace approximations

Abstract Aiming to deliver improved precipitation simulations for hydrological impact assessment studies, we develop a methodology for modelling and simulating high-dimensional spatial precipitation extremes, focusing on both their marginal distributions and tail dependence structures. Tail dependence is crucial for assessing the consequences of extreme precipitation events, yet most stochastic weather generators do not attempt to capture this property. The spatial distribution of precipitation occurrences is modelled with four competing models, while the spatial distribution of nonzero extreme precipitation intensities are modelled with a latent Gaussian version of the spatial conditional extremes model. Nonzero precipitation marginal distributions are modelled using latent Gaussian models with gamma and generalized Pareto likelihoods. Fast inference is achieved using integrated nested Laplace approximations. We model and simulate spatial precipitation extremes in Central Norway, using 13 years of hourly radar data with a spatial resolution of 1×1km2, over an area of size 6,461km2, to describe the behaviour of extreme precipitation over a small drainage area. Inference on this high-dimensional data set is achieved within hours, and the simulations capture the main trends of the observed precipitation well.

Feature selection, construction and search space reduction based on genetic programming for high-dimensional datasets

Feature selection, construction and search space reduction based on genetic programming for high-dimensional datasets