Dimensionality Reduction Techniques Research Articles

Comparison of methods to aggregate climate data to predict crop yield: an application to soybean

High-dimensional climate data collected on a daily, monthly, or seasonal time step are now commonly used to predict crop yields worldwide with standard statistical models or machine learning models. Since the use of all available individual climate variables generally leads to calculation problems, over-fitting, and over-parameterization, it is necessary to aggregate the climate data used as predictors. However, there is no consensus on the best way to perform this task, and little is known about the impacts of the type of aggregation method used and of the temporal resolution of weather data on model performances. Based on historical data from 1981 to 2016 of soybean yield and climate on 3447 sites worldwide, this study compares different temporal resolutions (daily, monthly, or seasonal) and dimension reduction techniques (principal component analysis (PCA), partial least square regression, and their functional counterparts) to aggregate climate data used as inputs of machine learning and linear regression (LR) models predicting yields. Results showed that random forest models outperformed and were less sensitive to climate aggregation methods than LRs when predicting soybean yields. With our models, the use of daily climate data did not improve predictive performance compared to monthly data. Models based on PCA or averages of monthly data showed better predictive performance compared to those relying on more sophisticated dimension reduction techniques. By highlighting the high sensitivity of projected impact of climate on crop yields to the temporal resolution and aggregation of climate input data, this study reveals that model performances can be improved by choosing the most appropriate time resolution and aggregation techniques. Practical recommendations are formulated in this article based on our results.

A comparative study of feature selection and feature extraction methods for financial distress identification

Financial distress identification remains an essential topic in the scientific literature due to its importance for society and the economy. The advancements in information technology and the escalating volume of stored data have led to the emergence of financial distress that transcends the realm of financial statements and its’ indicators (ratios). The feature space could be expanded by incorporating new perspectives on feature data categories such as macroeconomics, sectors, social, board, management, judicial incident, etc. However, the increased dimensionality results in sparse data and overfitted models. This study proposes a new approach for efficient financial distress classification assessment by combining dimensionality reduction and machine learning techniques. The proposed framework aims to identify a subset of features leading to the minimization of the loss function describing the financial distress in an enterprise. During the study, 15 dimensionality reduction techniques with different numbers of features and 17 machine-learning models were compared. Overall, 1,432 experiments were performed using Lithuanian enterprise data covering the period from 2015 to 2022. Results revealed that the artificial neural network (ANN) model with 30 ranked features identified using the Random Forest mean decreasing Gini (RF_MDG) feature selection technique provided the highest AUC score. Moreover, this study has introduced a novel approach for feature extraction, which could improve financial distress classification models.

Anomaly Detection in Blockchain Using Machine Learning

Blockchain technology has gained significant attention as a secure and decentralized platform for various applications. However, the immutable and distributed nature of blockchain also presents unique challenges for detecting anomalies and suspicious activities within the network. This research paper proposes a novel approach to anomaly detection in blockchain using machine learning techniques.  The goal of this study is to develop an effective and scalable anomaly detection framework that can analyze the vast amount of data generated within a blockchain network and identify irregularities or potential security threats. The proposed framework leverages the power of machine learning algorithms to learn patterns, relationships, and behaviours from historical blockchain data, enabling the detection of anomalous activities in real time.The research paper first focuses on feature extraction techniques tailored specifically for blockchain data. These techniques consider key characteristics of blockchain transactions, such as transaction size, timestamp, and involved addresses, to construct meaningful features that capture the underlying patterns and trends. Various dimensionality reduction techniques are also explored to handle the high-dimensional nature of blockchain data.Subsequently, several machine learning algorithms, including clustering, classification, and anomaly detection methods, are employed to train models using the extracted features. The performance of different algorithms is evaluated using benchmark datasets and real-world blockchain data to assess their accuracy, precision, and recall in detecting anomalies. Additionally, the scalability of the proposed framework is investigated to ensure its effectiveness in large-scale blockchain networks.Furthermore, the research paper investigates the integration of domain-specific knowledge, such as known attack patterns and regulatory compliance rules, into the anomaly detection framework. This hybrid approach combines the strengths of machine learning algorithms with expert knowledge to enhance the accuracy and interpretability of anomaly detection results.The experimental results demonstrate that the proposed anomaly detection framework achieves promising performance in identifying various types of anomalies in blockchain data. It exhibits high detection rates while minimizing false positives, thereby providing valuable insights for blockchain network administrators and regulators to mitigate security risks and safeguard the integrity of blockchain systems. In conclusion, this research paper presents an innovative approach to anomaly detection in blockchain using machine learning. The proposed framework addresses the unique challenges posed by blockchain's decentralized and immutable nature, offering an effective solution for detecting suspicious activities and ensuring the security of blockchain networks. The findings of this study contribute to the growing field of blockchain analytics and have significant implications for real-world blockchain applications in domains such as finance, supply chain management, and healthcare.

TailorMe: Self‐Supervised Learning of an Anatomically Constrained Volumetric Human Shape Model

AbstractHuman shape spaces have been extensively studied, as they are a core element of human shape and pose inference tasks. Classic methods for creating a human shape model register a surface template mesh to a database of 3D scans and use dimensionality reduction techniques, such as Principal Component Analysis, to learn a compact representation. While these shape models enable global shape modifications by correlating anthropometric measurements with the learned subspace, they only provide limited localized shape control. We instead register a volumetric anatomical template, consisting of skeleton bones and soft tissue, to the surface scans of the CAESAR database. We further enlarge our training data to the full Cartesian product of all skeletons and all soft tissues using physically plausible volumetric deformation transfer. This data is then used to learn an anatomically constrained volumetric human shape model in a self‐supervised fashion. The resulting TailorMe model enables shape sampling, localized shape manipulation, and fast inference from given surface scans.

Measuring Regional Competitiveness: New Insights Based on the RCI

The design of a comprehensive measure of regional competitiveness has attracted much attention, with the Regional Competitiveness Index (RCI) as a benchmark. We contribute from two different perspectives. On the one hand, we test the robustness of the RCI results by modifying the dimensionality reduction technique. On the other hand, we explore the importance of two factors in the analysis of regional competitiveness: geographical distance to potential competitors (by correcting the index with the introduction of several distance measures) and technological specialization (by making a prior selection of the regions that are expected to compete mostly either in high-tech sectors or in medium- or low-tech sectors, and recalculating the index accordingly). The results show that the RCI indicator is robust to the variable reduction methods employed. Moreover, including geographical distance substantially modifies the degree of competitiveness of many regions, which we believe cannot be neglected since competition tends to be fiercer the shorter the distance between the regions involved. Finally, if regions are categorized into two groups based on their technology level, some changes in RCI rankings are very noticeable.

Multiple Disease Prediction System Using ML

Abstract: The increasing prevalence of diverse diseases presents a challenge to global healthcare systems, underscoring the need for innovative and efficient methods for early detection and preventive measures. This paper explores the application of machine learning algorithms in multiple disease prediction to enhance diagnostic accuracy and enable timely intervention. Leveraging diverse health-related data sources, including medical records and genomic information, comprehensive predictive models are developed. A multi-faceted machine learning approach integrates support vector machines, decision trees, neural networks, and ensemble learning methods to analyze complex data patterns. Feature selection and dimensionality reduction techniques optimize model performance and interpretability. The development of a predictive system involves essential steps such as data collection, preprocessing, and model training, followed by evaluation using metrics like accuracy and recall. Integration of Flask for web application development facilitates user interaction and prediction functionality. Deployment, testing, debugging, and ongoing maintenance ensure system efficiency and compliance with regulatory requirements for healthcare data security and privacy.

Comprehensive analysis of human monocyte subsets using full-spectrum flow cytometry and hierarchical marker clustering

IntroductionExploring monocytes’ roles within the tumor microenvironment is crucial for crafting targeted cancer treatments.MethodsThis study unveils a novel methodology utilizing four 20-color flow cytometry panels for comprehensive peripheral immune system phenotyping, specifically targeting classical, intermediate, and non-classical monocyte subsets. ResultsBy applying advanced dimensionality reduction techniques like t-distributed stochastic neighbor embedding (tSNE) and FlowSom analysis, we performed an extensive profiling of monocytes, assessing 50 unique cell surface markers related to a wide range of immunological functions, including activation, differentiation, and immune checkpoint regulation. DiscussionThis in-depth approach significantly refines the identification of monocyte subsets, directly supporting the development of personalized immunotherapies and enhancing diagnostic precision. Our pioneering panel for monocyte phenotyping marks a substantial leap in understanding monocyte biology, with profound implications for the accuracy of disease diagnostics and the success of checkpoint-inhibitor therapies. Key findings include revealing distinct marker expression patterns linked to tumor progression and providing new avenues for targeted therapeutic interventions.

Proper Orthogonal Decomposition Based Response Analysis of Inlet Distortion on a Waterjet Pump

This study addresses the challenge of performance degradation in waterjet pumps due to non-uniform suction flow. Utilizing the Proper Orthogonal Decomposition (POD) method, it decomposes and reconstructs the flow features within a waterjet pump under non-uniform inflow into a series of modes ranked in descending order of energy. By analyzing the modes with dominant energy, which contain complex information about the flow field, it is revealed that modes 1 and 2 predominantly represent the formation of a concentrated vortex, whereas modes 3 and 4 illustrate its spatial offset. Notably, in the hub section, mode 3 exhibits a delayed flow separation caused by the reduction of circumferential vortex (CV), with a consequent lift in blade loading at the leading edge and a higher head compared to mode 1. In the shroud section, the delayed flow separation in mode 3 suppressed reverse flow and the concentrated separation vortex (CSV) and then increased the blade loading, ultimately enhancing the pump head. The findings provide significant insights into optimizing waterjet pump performance by detailing the interactions between various flow structures and pump components, effectively filling a knowledge gap in applying dimensionality reduction techniques within the distorted flow fields of water jet pumps.

Machine Learning and Deep Learning Sentiment Analysis Models: Case Study on the SENT-COVID Corpus of Tweets in Mexican Spanish

This article presents a comprehensive evaluation of traditional machine learning and deep learning models in analyzing sentiment trends within the SENT-COVID Twitter corpus, curated during the COVID-19 pandemic. The corpus, filtered by COVID-19 related keywords and manually annotated for polarity, is a pivotal resource for conducting sentiment analysis experiments. Our study investigates various approaches, including classic vector-based systems such as word2vec, doc2vec, and diverse phrase modeling techniques, alongside Spanish pre-trained BERT models. We assess the performance of readily available sentiment analysis libraries for Python users, including TextBlob, VADER, and Pysentimiento. Additionally, we implement and evaluate traditional classification algorithms such as Logistic Regression, Naive Bayes, Support Vector Machines, and simple neural networks like Multilayer Perceptron. Throughout the research, we explore different dimensionality reduction techniques. This methodology enables a precise comparison among classification methods, with BETO-uncased achieving the highest accuracy of 0.73 on the test set. Our findings underscore the efficacy and applicability of traditional machine learning and deep learning models in analyzing sentiment trends within the context of low-resource Spanish language scenarios and emerging topics like COVID-19.

Balancing Techniques for Advanced Financial Distress Detection Using Artificial Intelligence

Imbalanced datasets are one of the main issues encountered by artificial intelligence researchers, as machine learning (ML) algorithms can become biased toward the majority class and perform insufficiently on the minority classes. Financial distress (FD) is one of the numerous real-world applications of ML, struggling with this issue. Furthermore, the topic of financial distress holds considerable interest for both academics and practitioners due to the non-determined indicators of condition states. This research focuses on the involvement of balancing techniques according to different FD condition states. Moreover, this research was expanded by implementing ML models and dimensionality reduction techniques. During the course of this study, a Combined FD was constructed using five distinct conditions, ten distinct class balancing techniques, five distinct dimensionality reduction techniques, two features selection strategies, eleven machine learning models, and twelve weighted majority algorithms (WMAs). Results revealed that the highest area under the receiver operating characteristic (ROC) curve (AUC) score was achieved when using the extreme gradient boosting machine (XGBoost) feature selection technique, the experimental max number strategy, the undersampling methods, and the WMA 3.1 weighted majority algorithm (i.e., with categorical boosting (CatBoost), XGBoost, and random forest (RF) having equal voting weights). Moreover, this research has introduced a novel approach for setting the condition states of financial distress, including perspectives from debt and change in employment. These outcomes have been achieved utilizing authentic enterprise data from small and medium Lithuanian enterprises.

Probabilistic assessment of scalar transport under hydrodynamically unstable flows in heterogeneous porous media

Quantitative predictions of scalar transport in natural porous media is a nontrivial task given the presence of multi-scale spatial heterogeneity in the permeability field. Due to data scarcity, the structural map of the permeability field is subject to uncertainty and therefore, model predictions are uncertain. For such reasons, probabilistic models of flow and transport in natural porous media are required in risk assessment and to provide reliable decision making under uncertainty. Further complexities arise when the viscosity of the injected solute differs from that of the ambient fluid. Under the presence of viscosity contrast, hydrodynamic instabilities give rise to viscous fingering, which induces additional disorder in both velocity and solute concentration fields. This work examines the combined role of viscous fingering and permeability heterogeneity in the probabilistic description of transport predictions. In particular, we focus on metrics that are important for risk analysis, such as the solute plume’s early arrival times and the maximum concentration observed at a given location. We propose to use the Projection Pursuit Adaptation (PPA) method in the Polynomial Chaos Expansion (PCE) framework to quantify uncertainty in transport model predictions. The PPA method is a data-driven approach that optimally represents a given quantity of interest in a low-dimensional manifold. Unlike other dimension reduction techniques in uncertainty quantification, the PPA method utilizes non-linear information of the quantity of interest to identify the low-dimensional manifold, thereby increasing the likelihood of finding a more accurate lower-dimensional space. Moreover, the PPA model converges to the physical solution in a mean squared sense with respect to the polynomial order, enabling the construction of a converged model even with limited available data. Then, the PPA results are compared to Monte Carlo simulations using the same amount of data. This comparison illustrates that while Monte Carlo simulations are able to capture low-order statistics, they struggle to represent more detailed probability density functions. Our results show how the combined effect of permeability heterogeneity and viscosity contrast can enhance the mobility of the solute plume.

Dynamic sparse PCA: a dimensional reduction method for sensor data in virtual metrology

Virtual metrology (VM) is the ideal technique for predicting the quality of every wafer and reducing overall inspection costs in semiconductor manufacturing. The data from numerous monitoring sensors in semiconductor production poses challenges in terms of data dimensionality. Thus, dimensionality reduction techniques are required before building the prediction model in VM. Additionally, to reduce the sensor cost, it is crucial to determine the sensors that contribute significantly to the output of the prediction model. Therefore, in this paper, we propose a dimensionality reduction technique called dynamic sparse principal component analysis to establish global sparsity in the first few principal components. The proposed method seeks extracted features that involve a limited number of sensors in the VM to reduce sensor cost and increase the interpretability of the features while reasonably capturing data variability. The experimental results demonstrate that the proposed method is not sensitive to the scale of the problem and can solve the large-scale high-dimensional problems that are commonly encountered in semiconductor manufacturing.

Data Efficiency, Dimensionality Reduction, and the Generalized Symmetric Information Bottleneck.

The symmetric information bottleneck (SIB), an extension of the more familiar information bottleneck, is a dimensionality-reduction technique that simultaneously compresses two random variables to preserve information between their compressed versions. We introduce the generalized symmetric information bottleneck (GSIB), which explores different functional forms of the cost of such simultaneous reduction. We then explore the data set size requirements of such simultaneous compression. We do this by deriving bounds and root-mean-squared estimates of statistical fluctuations of the involved loss functions. We show that in typical situations, the simultaneous GSIB compression requires qualitatively less data to achieve the same errors compared to compressing variables one at a time. We suggest that this is an example of a more general principle that simultaneous compression is more data efficient than independent compression of each of the input variables.

A computational topology-based spatiotemporal analysis technique for honeybee aggregation

A primary challenge in understanding collective behavior is characterizing the spatiotemporal dynamics of the group. We employ topological data analysis to explore the structure of honeybee aggregations that form during trophallaxis, which is the direct exchange of food among nestmates. From the positions of individual bees, we build topological summaries called CROCKER matrices to track the morphology of the group as a function of scale and time. Each column of a CROCKER matrix records the number of topological features, such as the number of components or holes, that exist in the data for a range of analysis scales, at a given point in time. To detect important changes in the morphology of the group from this information, we first apply dimensionality reduction techniques to these matrices and then use classic clustering and change-point detection algorithms on the resulting scalar data. A test of this methodology on synthetic data from an agent-based model of honeybees and their trophallaxis behavior shows two distinct phases: a dispersed phase that occurs before food is introduced, followed by a food-exchange phase during which aggregations form. We then move to laboratory data, successfully detecting the same two phases across multiple experiments. Interestingly, our method reveals an additional phase change towards the end of the experiments, suggesting the possibility of another dispersed phase that follows the food-exchange phase.

Developing a hierarchical model for unraveling conspiracy theories

A conspiracy theory (CT) suggests covert groups or powerful individuals secretly manipulate events. Not knowing about existing conspiracy theories could make one more likely to believe them, so this work aims to compile a list of CTs shaped as a tree that is as comprehensive as possible. We began with a manually curated ‘tree’ of CTs from academic papers and Wikipedia. Next, we examined 1769 CT-related articles from four fact-checking websites, focusing on their core content, and used a technique called Keyphrase Extraction to label the documents. This process yielded 769 identified conspiracies, each assigned a label and a family name. The second goal of this project was to detect whether an article is a conspiracy theory, so we built a binary classifier with our labeled dataset. This model uses a transformer-based machine learning technique and is pre-trained on a large corpus called RoBERTa, resulting in an F1 score of 87%. This model helps to identify potential conspiracy theories in new articles. We used a combination of clustering (HDBSCAN) and a dimension reduction technique (UMAP) to assign a label from the tree to these new articles detected as conspiracy theories. We then labeled these groups accordingly to help us match them to the tree. These can lead us to detect new conspiracy theories and expand the tree using computational methods. We successfully generated a tree of conspiracy theories and built a pipeline to detect and categorize conspiracy theories within any text corpora. This pipeline gives us valuable insights through any databases formatted as text.

Multidimensional scaling for big data

AbstractWe present a set of algorithms implementing multidimensional scaling (MDS) for large data sets. MDS is a family of dimensionality reduction techniques using a $$n \times n$$ n × n distance matrix as input, where n is the number of individuals, and producing a low dimensional configuration: a $$n\times r$$ n × r matrix with $$r<<n$$ r < < n . When n is large, MDS is unaffordable with classical MDS algorithms because their extremely large memory and time requirements. We compare six non-standard algorithms intended to overcome these difficulties. They are based on the central idea of partitioning the data set into small pieces, where classical MDS methods can work. Two of these algorithms are original proposals. In order to check the performance of the algorithms as well as to compare them, we have done a simulation study. Additionally, we have used the algorithms to obtain an MDS configuration for EMNIST: a real large data set with more than 800000 points. We conclude that all the algorithms are appropriate to use for obtaining an MDS configuration, but we recommend to use one of our proposals, since it is a fast algorithm with satisfactory statistical properties when working with big data. An package implementing the algorithms has been created.

Clustering of lifestyle risk factors in relation to suicidal thoughts and behaviors in young adolescents: a cross-national study of 45 low- and middle-income countries

BackgroundPrior research has reaffirmed lifestyle risk behaviors to cluster among adolescents. However, the lifestyle cluster effect on suicidal thoughts and behaviors (STBs) was unclear among adolescents in low- and middle-income countries (LMICs). No comparison of such associations was conducted across nations.MethodsData from 45 LMICs were obtained from the Global School-based Student Health Survey (GSHS) between 2009 and 2019. Lifestyle behavior factors were collected through a structured questionnaire. Suicidal ideation, plan, and attempt were ascertained by three single-item questions. Lifestyle risk scores were calculated via a sufficient dimension reduction technique, and lifestyle risk clusters were constructed using a latent class analysis. Generalized linear mixed models with odds ratio (OR) and 95% confidence interval (CI) were used to estimate the lifestyle-STB associations.ResultsA total of 229,041 adolescents were included in the final analysis. The weighted prevalence of suicidal ideation, plan, and attempt was 7.37%, 5.81%, and 4.59%, respectively. Compared with the favorable lifestyle group, the unfavorable group had 1.48-, 1.53-, and 3.11-fold greater odds of suicidal ideation (OR = 1.48, 95%CI: 1.30–1.69), plan (OR = 1.53, 95%CI 1.34–1.75), and attempt (OR = 3.11, 95%CI 2.64–3.65). Four clusters of lifestyle risk behaviors were identified, namely healthy lifestyles (H–L), insufficient intake of vegetables and fruit (V-F), frequent consumption of soft drinks and fast food (D-F), and tobacco smoking and alcohol drinking (S-A) clusters. Compared with H–L cluster, V-F cluster was associated with 43% and 42% higher odds of suicidal ideation and plan, followed by S-A cluster (26% for ideation and 20% for plan), but not significant in D-F cluster (P > 0.05). D-F cluster was associated with 2.85-fold increased odds of suicidal attempt, followed by V-F cluster (2.43-fold) and S-A cluster (1.18-fold).ConclusionsClustering of lifestyle risk behaviors is informative for risk stratification of STBs in resource-poor settings. Lifestyle-oriented suicide prevention efforts should be initiated among school-attending adolescents in LMICs.

Enhanced Model Predictions through Principal Components and Average Least Squares-Centered Penalized Regression

We address the estimation of regression parameters for the ill-conditioned predictive linear model in this study. Traditional least squares methods often encounter challenges in yielding reliable results when there is multicollinearity. Therefore, we employ a better shrinkage method, average least squares-centered penalized regression (ALPR), as it offers a more efficient approach for handling multicollinearity than ridge regression. Additionally, we integrate ALPR with the principal component (PC) dimension reduction method for enhanced performance. We compared the proposed PCALPR estimation technique with existing ones for ill-conditioned problems through comprehensive simulations and real-life data analyses using the mean squared error. This integration results in superior model performance compared to other methods, highlighting the potential of combining dimensionality reduction techniques with penalized regression for enhanced model predictions.

NSimplex Zen : A Novel Dimensionality Reduction for Euclidean and Hilbert Spaces

Dimensionality reduction techniques map values from a high dimensional space to one with a lower dimension. The result is a space which requires less physical memory and has a faster distance calculation. These techniques are widely used where required properties of the reduced-dimension space give an acceptable accuracy with respect to the original space. Many such transforms have been described. They have been classified in two main groups: linear and topological . Linear methods such as Principal Component Analysis (PCA) and Random Projection (RP) define matrix-based transforms into a lower dimension of Euclidean space. Topological methods such as Multidimensional Scaling (MDS) attempt to preserve higher-level aspects such as the nearest-neighbour relation, and some may be applied to non-Euclidean spaces. Here, we introduce nSimplex Zen , a novel topological method of reducing dimensionality. Like MDS, it relies only upon pairwise distances measured in the original space. The use of distances, rather than coordinates, allows the technique to be applied to both Euclidean and other Hilbert spaces, including those governed by Cosine, Jensen–Shannon and Quadratic Form distances. We show that in almost all cases, due to geometric properties of high-dimensional spaces, our new technique gives better properties than others, especially with reduction to very low dimensions.

LaplaceConfidence: A graph-based approach for learning with noisy labels

Real-world machine learning applications seldom provide perfect labeled data, posing a challenge in developing models robust to noisy labels. Recent methods prioritize noise filtering based on the discrepancies between model predictions and the provided noisy labels, assuming samples with minimal classification losses to be clean. In this work, we capitalize on the consistency between the learned model and the complete noisy dataset, employing the data’s rich representational and topological information. We introduce LaplaceConfidence, a method that to obtain label confidence (i.e., clean probabilities) utilizing the Laplacian energy. Specifically, it first constructs graphs based on the feature representations of all noisy samples and minimizes the Laplacian energy to produce a low-energy graph. Clean labels should fit well into the low-energy graph while noisy ones should not, allowing our method to determine data’s clean probabilities. Furthermore, LaplaceConfidence is embedded into a holistic method for robust training, where co-training technique generates unbiased label confidence and label refurbishment technique better utilizes it. We also explore the dimensionality reduction technique to accommodate our method on large-scale noisy datasets. Our experiments demonstrate that LaplaceConfidence outperforms state-of-the-art methods on benchmark datasets under both synthetic and real-world noise. Code available at https://github.com/chenmc1996/LaplaceConfidence.