Hierarchical Machine Learning Research Articles

Inferring ancestry with the hierarchical soft clustering approach tangleGen.

Understanding the genetic ancestry of populations is central to numerous scientific and societal fields. It contributes to a better understanding of human evolutionary history, advances personalized medicine, aids in forensic identification, and allows individuals to connect to their genealogical roots. Existing methods, such as ADMIXTURE, have significantly improved our ability to infer ancestries. However, these methods typically work with a fixed number of independent ancestral populations. As a result, they provide insight into genetic admixture, but do not include a hierarchical interpretation. In particular, the intricate ancestral population structures remain difficult to unravel. Alternative methods with a consistent inheritance structure, such as hierarchical clustering, may offer benefits in terms of interpreting the inferred ancestries. Here, we present tangleGen, a soft clustering tool that transfers the hierarchical machine learning framework Tangles, which leverages graph theoretical concepts, to the field of population genetics. The hierarchical perspective of tangleGen on the composition and structure of populations improves the interpretability of the inferred ancestral relationships. Moreover, tangleGen adds a new layer of explainability, as it allows identifying the SNPs that are responsible for the clustering structure. We demonstrate the capabilities and benefits of tangleGen for the inference of ancestral relationships, using both simulated data and data from the 1000 Genomes Project.

A Hierarchical Machine Learning Method for Detection and Visualization of Network Intrusions from Big Data

Machine learning is regarded as an effective approach in network intrusion detection, and has gained significant attention in recent studies. However, few intrusion detection methods have been successfully applied to detect anomalies in large-scale network traffic data, and low explainability of the complex algorithms has caused concerns about fairness and accountability. A further problem is that many intrusion detection systems need to work with distributed data sources in the cloud. In this paper, we propose an intrusion detection method based on distributed computing to learn the latent representations from large-scale network data with lower computation time while improving the intrusion detection accuracy. Our proposed classifier, based on a novel hierarchical algorithm combining adaptability and visualization ability from a self-structured unsupervised learning algorithm and achieving explainability from self-explainable supervised algorithms, is able to enhance the understanding of the model and data. The experimental results show that our proposed method is effective, efficient, and scalable in capturing the network traffic patterns and detecting detailed network intrusion information such as type of attack with high detection performance, and is an ideal method to be applied in cloud-computing environments.

Advancing micromechanical property characterization in ceramic multilayer coatings via hierarchical machine learning

Advancing micromechanical property characterization in ceramic multilayer coatings via hierarchical machine learning

Exploring the Effects of Online Learning Complications on Mathematics Achievement

The aim of the research was to examine different complications of learning mathematics online and their effects on students’ academic performances. A cross-sectional survey design was employed among 2311 students from the school to university levels in Nepal. Hierarchical linear regression and machine learning were major statistical techniques used in the research. The findings indicated that access to and cost of digital devices, training on digital skills, assignment submission skills, digital literacy, and a separate study room at home were major complications for students in learning mathematics online. Additionally, these complications to learning mathematics online negatively affect students’ academic performance. This research gives new insights into the complications of online learning and its effect on learners’ performance in mathematics as well.

Information-extreme machine learning of an ophthalmic diagnostic system with a hierarchical class structure

The paper considers the method of hierarchical information-extreme machine learning for the system of ophthalmic diagnosis of eye pathology. Since the proposed method is developed within the framework of a functional approach to modeling the cognitive processes of natural intelligence, it, unlike neuro-like structures, acquires the properties of flexibility when retraining a diagnostic system and requires an order of magnitude fewer image samples. In addition, the decision rules based on the results of machine learning within the geometric approach in the form of a binary hierarchical structure of recognition classes ensure their practical invariance to the multidimensionality of both the space of diagnostic features and the alphabet of recognition classes. The modified Kullback-Leibler information measure, which is considered as a function of the accuracy of classification solutions, is chosen as a criterion for optimizing the parameters of the machine learning system for diagnosing eye pathologies. A hierarchical information-extreme machine learning algorithm for an ophthalmic diagnostic system for six eye pathologies was developed and programmatically implemented. Based on the results of functional diagnostics, it has been experimentally proved that the constructed decision rules are error-free according to the training matrices of recognition classes of each level of the constructed binary hierarchical structure.

High-accuracy full-coverage PM2.5 retrieval from 2014 to 2023 over China based on satellite remote sensing and hierarchical deep learning model

ABSTRACT Obtaining precise ground-level fine particulate matter (PM2.5) information is significant for human health. Spatial PM2.5 maps can be obtained by remote sensing technology, but considerable uncertainty exists when suffering from high pollution with complicated aerosol types. To address this issue, we propose using a hierarchical machine learning model to retrieve high-accuracy and daily full-coverage PM2.5 concentrations from 2014 to 2023 in China. Our hierarchical model was validated by the sample-based 10 cross-validation . Results suggest that our model performs better in terms of RMSE of 12.12 µg/m3, MAE of 8.14 µg/m3 and R2 of 0.95 than traditional model with RMSE of 18.18 µg/m3, MAE of 12.21 µg/m3 and R2 of 0.89, showing 27.49–37.41% improvements for RMSE, 21.85–39.26% improvements for MAE and 8.31–15.39% improvements for R2 at three-folds samples. On longer time scales, our model also shows better results than previous studies. Additionally, for high-pollution provinces, our model can capture PM2.5 trends more preciously than the traditional model. Under severe haze, our hierarchical model can also rightly reflect PM2.5 changes. Overall, due to the hierarchical strategy, our ML-based model can obtain daily full-coverage PM2.5 maps in China with high accuracy and can be applied for follow-up studies.

Large-scale prediction of solar irradiation, shading impacts, and energy generation on building Façade through urban morphological indicators: A machine learning approach

Incorporating Building-integrated Photovoltaics (BIPVs) into urban environments offers a sustainable approach by transforming building exteriors into renewable energy sources, especially by utilizing the extensive vertical surfaces in cities. While individual building applications have been extensively studied, there is a lack of research on BIPV potential at the urban scale due to the complexity of urban fabrics. Therefore, this study presents a novel method for predicting large-scale solar irradiation and energy generation on building surfaces using urban morphological indicators through machine learning models. Using an Australian city as a case study, a hierarchical two-level machine learning model is developed to predict the solar potential for building façade PV systems. The model incorporates 13 urban morphological predictors to determine average shading height and unshaded solar irradiance. By combining these predictions with wall datasets and different façade BIPV efficiencies, the proposed method predicts unshaded electricity generation. The results showcase high efficiency and accuracy in the prediction of the shading height, solar irradiance, and electricity generation. This innovative approach aids urban planners in considering large-scale surface BIPVs into urban development.

Hierarchical machine learning models can identify stimuli of climate change misinformation on social media

Misinformation about climate change poses a substantial threat to societal well-being, prompting the urgent need for effective mitigation strategies. However, the rapid proliferation of online misinformation on social media platforms outpaces the ability of fact-checkers to debunk false claims. Automated detection of climate change misinformation offers a promising solution. In this study, we address this gap by developing a two-step hierarchical model. The Augmented Computer Assisted Recognition of Denial and Skepticism (CARDS) model is specifically designed for categorising climate claims on Twitter. Furthermore, we apply the Augmented CARDS model to five million climate-themed tweets over a six-month period in 2022. We find that over half of contrarian climate claims on Twitter involve attacks on climate actors. Spikes in climate contrarianism coincide with one of four stimuli: political events, natural events, contrarian influencers, or convinced influencers. Implications for automated responses to climate misinformation are discussed.

Generation and deployment of honeytokens in relational databases for cyber deception

Despite considerable investments in database security, global statistics indicate an exponential increase in data breaches. Organizations are often unaware of data breaches for weeks, months, or even years. Sufficient for adversaries to compromise and ex-filtrate business or mission-critical data. Recent research suggests using honeytokens for early detection of data breaches in organizations. Existing honeytoken generation methods rely on regular expressions, rule mining, constraint satisfaction, or representation learning, which are complex and limited to a few attributes. We created a framework for generating and deploying honeytokens in relational databases that actively monitor sensitive records and quickly detect data breaches and their misuse. To generate the honeytoken we have used the hierarchical machine learning algorithm which uses a recursive technique to model the parent–child relationships of multi-table databases. The proposed method enables the organization to take remedial action to reduce the impact of data breaches and complement existing database security solutions.

Prediction of Steam Turbine Blade Erosion Using Computational Fluid Dynamics Simulation Data and Hierarchical Machine Learning

Abstract The information of the degree of blade erosion is vital for the efficient operation of steam turbines. However, it is nearly impossible to directly measure the degree of blade erosion during operation. Moreover, collecting sufficient data of eroded cases for predictive analysis is challenging. Therefore, this paper proposes a blade erosion prediction method using numerical simulation and machine learning. Pressure data of several blade erosion cases are collected from the numerical turbine simulation. The machine learning approach involves training on collected simulation data to predict the degree of erosion for the first-stage stator (1S) and the first-stage rotor blade (1R) from internal pressure data. The proposed erosion prediction model employs a two-step hierarchical approach. First, the proposed model predicts the 1S erosion degree using the k-nearest neighbor (k-NN) regression. Second, the proposed model estimates the 1R erosion degree with linear regression models. These models are tailored for each of the 1S erosion degrees, utilizing pressure data processed through fast Fourier transform (FFT). The evaluation shows that the proposed method achieves the prediction of the 1S erosion with a mean absolute error (MAE) of 0.000693 mm and the 1R erosion with an MAE of 0.458 mm. The evaluation results indicate that the proposed method can accurately capture the degree of turbine blade erosion from internal pressure data. As a result, the proposed method suggests that the erosion prediction method can be effectively used to determine the optimal timing for maintenance, repair, and overhaul (MRO).

Hitac: a hierarchical taxonomic classifier for fungal ITS sequences compatible with QIIME2

BackgroundFungi play a key role in several important ecological functions, ranging from organic matter decomposition to symbiotic associations with plants. Moreover, fungi naturally inhabit the human body and can be beneficial when administered as probiotics. In mycology, the internal transcribed spacer (ITS) region was adopted as the universal marker for classifying fungi. Hence, an accurate and robust method for ITS classification is not only desired for the purpose of better diversity estimation, but it can also help us gain a deeper insight into the dynamics of environmental communities and ultimately comprehend whether the abundance of certain species correlate with health and disease. Although many methods have been proposed for taxonomic classification, to the best of our knowledge, none of them fully explore the taxonomic tree hierarchy when building their models. This in turn, leads to lower generalization power and higher risk of committing classification errors.ResultsHere we introduce HiTaC, a robust hierarchical machine learning model for accurate ITS classification, which requires a small amount of data for training and can handle imbalanced datasets. HiTaC was thoroughly evaluated with the established TAXXI benchmark and could correctly classify fungal ITS sequences of varying lengths and a range of identity differences between the training and test data. HiTaC outperforms state-of-the-art methods when trained over noisy data, consistently achieving higher F1-score and sensitivity across different taxonomic ranks, improving sensitivity by 6.9 percentage points over top methods in the most noisy dataset available on TAXXI.ConclusionsHiTaC is publicly available at the Python package index, BIOCONDA and Docker Hub. It is released under the new BSD license, allowing free use in academia and industry. Source code and documentation, which includes installation and usage instructions, are available at https://gitlab.com/dacs-hpi/hitac.

Comparative Study of the Equal-Weight Method and Hierarchical Risk Parity in Portfolio Construction

Purpose: Portfolio optimization is a process in which the capital is allocated among the portfolio assets such that the return is maximized while the risk is minimized. Portfolio construction and optimization has long been an active research area in finance. For the portfolios with highly correlated assets, the performance of traditional risk-based asset allocation methods such as, the mean-variance (MV) method is limited because quadratic optimizers require an inversion of the covariance matrix of the portfolio to distribute weight among the portfolio assets. Methods: A possible solution to the limitations of traditional risk-based asset allocation methods can be provided by a hierarchical clustering-based Machine Learning method because it uses hierarchical relationships between the covariance of assets in the portfolio to distribute the weight, and inversion of the covariance matrix is not required. A comparison of the performance of a simple non-optimization technique called the Equal-weight (EW) method to the two optimization methods, the Mean-variance method and the HRP method, which is a machine learning method, was conducted in this research. Results: It was found that in terms of cumulative returns, the equal-weight method has outperformed several more sophisticated optimization techniques, the mean-variance method, and the HRP method. For most of the period, the Sharpe ratio of the HRP method was observed to be similar to the mean-variance method and equal-weight method. Implications: This research supports the idea that HRP is a feasible method to construct portfolios with correlated assets because the performance of HRP is comparable to the performances of the traditional optimization method and the non-optimization method.

On the importance of integrating convolution features for Indian medicinal plant species classification using hierarchical machine learning approach

This work proposes a novel hierarchical classification framework designed to categorize hundred Indian medicinal plant species. The innovation lies in introducing a comprehensive feature representation by integrating convolutional features with geometric, texture, shape, and multispectral features for classification tasks. In this study, a two-level hierarchical plant classification model is proposed to address the challenges of inter-class similarity and intra-class variations. The first level classifies 100 medicinal plant species into 11 groups based on visual similarities among the plants. At level two, the specific plant species containing in each group are predicted using Random Forest classifier. The evaluation is performed at two levels to analyze the effectiveness of the proposed model. The performance analysis compares the effectiveness of individual feature types against the composite feature model. Performance is also evaluated based on specific groups that demonstrate high similarity between classes and intra-class variations among the plant species separately. Furthermore, the generality of the model is tested using two self-created datasets-RTL80 and RTP40, requiring more than 300 man-hours to collect. Experimental results demonstrate a promising accuracy of 94.54% on GSL100 leaf dataset and 75.46% on RTL80 and RTP40 real-time datasets reflecting the superiority of the proposed hierarchical model over state-of-the-art methods.

Rising summer river water temperature across Canada: spatial patterns and hydroclimatic controls

Understanding the spatio-temporal variability of climate-induced river water temperature change is critical for identifying hotspots and assessing the impacts on ecological and socioeconomic systems. Here, we employ the air2stream model reconstructed river temperature records for 106 stations in Canada (Nash Sutcliffe coefficient goodness-of-fit: minimum = 0.79; median = 0.93; maximum = 0.97) to analyze summer temperature changes over the years 1980–2018. Results reveal widespread river temperature increases from June to September, with significantly increasing trends for about 40%–60% of stations. Additionally, we find significantly rising 7-day maximum temperature and increasing occurrences over the critical 18 and 20 °C thresholds for about 30%–65% of stations. Furthermore, by employing the Ward’s agglomerative hierarchical clustering machine learning (ML) method, we identify eight regions of spatially coherent variability and change. We find that the south-east, coast and northern prairies are the regions of high vulnerability because of the likely impacts of rising summer water temperatures on cold-water aquatic species. Additionally, by using the random forests ML method, we demonstrate that mean air temperature and its trends are the primary drivers of mean water temperature and trends, respectively. Thus, with the projected enhanced air temperature increase across Canada, an amplified future summer river warming can be expected, which could have severe consequences, particularly in already thermally-stressed river systems.

An integrated hierarchical classification and machine learning approach for mapping land use and land cover in complex social-ecological systems

Mapping land use and land cover (LULC) using remote sensing is fundamental to environmental monitoring, spatial planning and characterising drivers of change in landscapes. We develop a new, general and versatile approach for mapping LULC in landscapes with relatively gradual transition between LULC categories such as African savannas. The approach integrates a well-tested hierarchical classification system with the computationally efficient random forest (RF) classifier and produces detailed, accurate and consistent classification of structural vegetation heterogeneity and density and anthropogenic land use. We use Landsat 8 OLI imagery to illustrate this approach for the Extended Greater Masai Mara Ecosystem (EGMME) in southwestern Kenya. We stratified the landscape into eight relatively homogeneous zones, systematically inspected the imagery and randomly allocated 1,697 training sites, 556 of which were ground-truthed, proportionately to the area of each zone. We directly assessed the accuracy of the visually classified image. Accuracy was high and averaged 88.1% (80.5%–91.7%) across all the zones and 89.1% (50%–100%) across all the classes. We applied the RF classifier to randomly selected samples from the original training dataset, separately for each zone and the EGMME. We evaluated the overall and class-specific accuracy and computational efficiency using the Out-of-Bag (OOB) error. Overall accuracy (79.3%–97.4%) varied across zones but was higher whereas the class-specific accuracy (25.4%–98.1%) was lower than that for the EGMME (80.2%). The hierarchical classifier identified 35 LULC classes which we aggregated into 18 intermediate mosaics and further into five more general categories. The open grassed shrubland (21.8%), sparse shrubbed grassland (10.4%) and small-scale cultivation (13.3%) dominated at the detailed level, grassed shrubland (31.9%) and shrubbed grassland (28.9%) at the intermediate level, and grassland (35.7%), shrubland (35.3%) and woodland (12.5%) at the general level. Our granular LULC map for the EGMME is sufficiently accurate for important practical purposes such as land use spatial planning, habitat suitability assessment and temporal change detection. The extensive ground-truthing data, sample site photos and classified maps can contribute to wider validation efforts at regional to global scales.

Optimization of number and range of shunt valve performance levels in infant hydrocephalus: a machine learning analysis.

Shunt surgery is the main treatment modality for hydrocephalus, the leading cause of brain surgery in children. The efficacy of shunt surgery, particularly in infant hydrocephalus, continues to present serious challenges in achieving improved outcomes. The crucial role of correct adjustments of valve performance levels in shunt outcomes has been underscored. However, there are discrepancies in the performance levels of valves from different companies. This study aims to address this concern by optimizing both the number and range of valve performance levels for infant hydrocephalus, aiming for improved shunt surgery outcomes. We conducted a single-center cohort study encompassing infant hydrocephalus cases that underwent initial shunt surgery without subsequent failure or unimproved outcomes. An unsupervised hierarchical machine learning method was utilized for clustering and reporting the valve drainage pressure values for all patients within each identified cluster. The optimal number of clusters corresponds to the number of valve performance levels, with the valve drainage pressure ranges within each cluster indicating the pressure range for each performance level. Comparisons based on the Silhouette coefficient between 3-7 clusters revealed that this coefficient for the 4-cluster (4-performance level) was at least 28.3% higher than that of other cluster formations in terms of intra-cluster similarity. The Davies-Bouldin index for the 4-performance level was at least 37.2% lower than that of other configurations in terms of inter-cluster dissimilarity. Cluster stability, indicated by a Jaccard index of 71% for the 4-performance level valve, validated the robustness, reliability, and repeatability of our findings. Our suggested optimized drainage pressure ranges for each performance level (1.5-5.0, 5.0-9.0, 9.0-15.0, and 15.0-18.0cm H2O) may potentially assist neurosurgeons in improving clinical outcomes for patients with shunted infantile hydrocephalus.

Hierarchical energy signatures using machine learning for operational visibility and diagnostics in automotive manufacturing

Manufacturing energy consumption data contains important process signatures required for operational visibility and diagnostics. These signatures may be of different temporal scales, ranging from monthly to sub-second resolutions. We introduce a hierarchical machine learning approach to identify automotive process signatures from paint shop electricity consumption data at varying temporal scales (weekly and daily). A Multi-Layer Perceptron (MLP), a Convolutional Neural Network (CNN), and Principal Component Analysis (PCA) combined with Logistic Regression (LR) are used for the analysis. We validate the utility of the developed algorithms with subject matter experts for (i) better operational visibility, and (ii) identifying energy saving opportunities.

Multi-scale bioimpedance flexible sensing with causal hierarchical machine learning for fish vitality evaluation under adversity stress

It is expected that waterless low-temperature stressful environments will induce stress responses in fish and affect their vitality. In this study, we developed a laser-activated, stretchable, highly conductive liquid metal (LM) based flexible sensor system for fish multi-scale bioimpedance detection. It has excellent conformability, electrical conductivity, bending and cyclic tensile stability. Meanwhile, test result showed that wireless power supply is a potential solution for realizing safe power supply for devices inside waterless low-temperature packages. In addition, a hierarchical regression model (GC-HRM) based on Granger causality was established. The result showed that tissue bioimpedance can induce changes in individual bioimpedance with unidirectional Granger causality. The R2 of the linear regression (LR), support vector regression (SVR) and artificial neural network (ANN) models under single-scale individual bioimpedance were 0.85, 0.90 and 0.78, respectively. By adding the multi-scale bioimpedance features, the R2 of the LR, SVR and ANN models were improved to 0.95, 1.00 and 0.98, respectively.

Overlapping yet dissociable contributions of superiority illusion features to Ponzo illusion strength and metacognitive performance

Humans are typically inept at evaluating their abilities and predispositions. People dismiss such a lack of metacognitive insight into their capacities while even enhancing (albeit illusorily) self-evaluation such that they should have more desirable traits than an average peer. This superiority illusion helps maintain a healthy mental state. However, the scope and range of its influence on broader human behavior, especially perceptual tasks, remain elusive. As belief shapes the way people perceive and recognize, the illusory self-superiority belief potentially regulates our perceptual and metacognitive performance. In this study, we used hierarchical Bayesian estimation and machine learning of signal detection theoretic measures to understand how the superiority illusion influences visual perception and metacognition for the Ponzo illusion. Our results demonstrated that the superiority illusion correlated with the Ponzo illusion magnitude and metacognitive performance. Next, we combined principal component analysis and cross-validated regularized regression (relaxed elastic net) to identify which superiority components contributed to the correlations. We revealed that the “extraversion” superiority dimension tapped into the Ponzo illusion magnitude and metacognitive ability. In contrast, the “honesty-humility” and “neuroticism” dimensions only predicted Ponzo illusion magnitude and metacognitive ability, respectively. These results suggest common and distinct influences of superiority features on perceptual sensitivity and metacognition. Our findings contribute to the accumulating body of evidence indicating that the leverage of superiority illusion is far-reaching, even to visual perception.

ChloroDBPFinder: Machine Learning-Guided Recognition of Chlorinated Disinfection Byproducts from Nontargeted LC-HRMS Analysis.

High-resolution mass spectrometry (HRMS) is a prominent analytical tool that characterizes chlorinated disinfection byproducts (Cl-DBPs) in an unbiased manner. Due to the diversity of chemicals, complex background signals, and the inherent analytical fluctuations of HRMS, conventional isotopic pattern (37Cl/35Cl), mass defect, and direct molecular formula (MF) prediction are insufficient for accurate recognition of the diverse Cl-DBPs in real environmental samples. This work proposes a novel strategy to recognize Cl-containing chemicals based on machine learning. Our hierarchical machine learning framework has two random forest-based models: the first layer is a binary classifier to recognize Cl-containing chemicals, and the second layer is a multiclass classifier to annotate the number of Cl present. This model was trained using ∼1.4 million distinctive MFs from PubChem. Evaluated on over 14,000 unique MFs from NIST20, this machine learning model achieved 93.3% accuracy in recognizing Cl-containing MFs (Cl-MFs) and 92.9% accuracy in annotating the number of Cl for Cl-MFs. Furthermore, the trained model was integrated into ChloroDBPFinder, a standalone R package for the streamlined processing of LC-HRMS data and annotating both known and unknown Cl-containing compounds. Tested on existing Cl-DBP data sets related to aspartame chlorination in tap water, our ChloroDBPFinder efficiently extracted 159 Cl-containing DBP features and tentatively annotated the structures of 10 Cl-DBPs via molecular networking. In another application of a chlorinated humic substance, ChloroDBPFinder extracted 79 high-quality Cl-DBPs and tentatively annotated six compounds. In summary, our proposed machine learning strategy and the developed ChloroDBPFinder provide an advanced solution to identifying Cl-containing compounds in nontargeted analysis of water samples. It is freely available on GitHub (https://github.com/HuanLab/ChloroDBPFinder).