Hierarchical Clustering Method Research Articles

A Comparative Approach to Financial Clustering Models: (A Study of the Companies Listed on Tehran Stock Exchange)

Data mining is known as one of the powerful tools in generating information and knowledge from raw data, and Clustering as one of the standard methods in data mining is a suitable method for grouping data in different clusters that helps to understand and analyze relationships. It is one of the essential issues in the field of investment, so by using stock market clustering, helpful information can be obtained to predict changes in stock prices of different companies and then on how to decide the correct number and shares in the portfolio to private investors and financial professionals' help. The purpose of this study is to cluster the companies listed on the Tehran stock exchange using three methods of K-means Clustering, Hierarchical clustering, and Affinity propagation clustering and compare these three methods with each other. To conduct this research, the adjusted price of 50 listed companies for the period 2019-07-01 to 2020-09-29 has been used. The evaluation results show that the obtained silhouette coefficient for K-means Clustering is higher and, therefore, better than other methods for stock exchange data. In the continuation of the research, calculating the co-integration of stock pairs that have the same co-movement with each other were identified, and finally, clusters were compiled using the t-SNE method.

IMPULSE OSCILLOMETRY IDENTIFIES DISTINCT ASTHMA PHENOTYPES ON CLUSTER ANALYSIS

IMPULSE OSCILLOMETRY IDENTIFIES DISTINCT ASTHMA PHENOTYPES ON CLUSTER ANALYSIS

Hierarchical Means Clustering

In the cluster analysis literature, there are several partitioning (non-hierarchical) methods for clustering multivariate objects based on model estimation. Distinct to these methods is the use of a system of n nested statistical models and the optimization of a loss function to best-fit a clustering model to observed data. Many hierarchical clustering methods are not model-based where hierarchy is obtained using a divisive or agglomerative greedy procedure. This paper aims to fill this gap by proposing a novel hierarchical cluster analysis methodology called Hierarchical Means Clustering. HMC produces a set of nested partitions with a centroid-based model estimated via least-squares by minimizing the total within-cluster deviance of the n partitions in the hierarchy. Hierarchical Means Clustering produces a hierarchy formed by n-1 nested partitions from 2 to n clusters with minimal total cluster deviance. Six real data examples are featured, and key links to k-means, Ward’s method, Bisecting k-means and model-based hierarchical agglomerative clustering methods are discussed.

A Deeper Analysis of the Hierarchical Clustering and Set Unionability-Based Data Union Method

A Deeper Analysis of the Hierarchical Clustering and Set Unionability-Based Data Union Method

Hereditary analysis and genotype × environment interaction effects on growth and yield components of Bambara groundnut (Vigna subterranea (L.) Verdc.) over multi-environments

This investigation was carried out to explore G × E interaction for yield and its associated attributes in 30 Bambara groundnut genotypes across four environments in tropical Malaysia. Such evaluations are essential when the breeding program's objective is to choose genotypes with broad adaption and yield potential. Studies of trait relationships, variance components, mean performance, and genetic linkage are needed by breeders when designing, evaluating, and developing selection criteria for improving desired characteristics in breeding programs. The evaluation of breeding lines of Bambara groundnut for high yield across a wide range of environments is important for long-term production and food security. Each site's experiment employed a randomized complete block design with three replicates. Data on vegetative and yield component attributes were recorded. The analysis of variance revealed that there were highly significant (p ≤ 0.01) differences among the 30 genotypes for all variables evaluated. A highly significant and positive correlation was identified between yield per hectare and dry seed weight (0.940), hundred seed weight (0.844), fresh pod weight (0.832), and total pod weight (0.750); the estimated correlation between dry weight of pods and seed yield was 1.0. The environment was more important than genotype and G × E in determining yield and yield components.A total of 49% variation is covered by PC1 (33.9%) and PC2 (15.1%) and the genotypes formed five distinct clusters based on Ward hierarchical clustering (WHC) method. The genotypes S5G1, S5G3, S5G5, S5G6, S5G8, S5G7, S5G2, S5G4, S5G10, S5G13, S5G11, and S5G14 of clusters I, II, and III were closest to the ideal genotype with superior yield across the environments. The PCA variable loadings revealed that an index based on dry pod weight, hundred seed weight, number of total pods and fresh pod weight could be used as a selection criteria to improve seed yield of Bambara groundnut.

Recognizing and Analyzing Private Car Commuters Using Big Data of Electronic Registration Identification of Vehicles

Private cars’ travel has been one of the main factors causing urban traffic congestion. Especially during morning and evening rush hours, private car commuters bring a significant burden to traffic. However, there is very little literature on them due to the lack of access to relevant data. A real-world dataset containing vehicle passing records of Electronic Registration Identification (ERI) of vehicles provides us with an opportunity to research private car commuters. We propose a regular behavior-based model to recognize private car commuters. In the model, a regular behavior-based definition of private car commuters is firstly proposed. Then, TDSP(Time dependent shortest path)-based distance measurement and a hierarchical clustering method are designed to extract regular behaviors. Furthermore, we utilize a regular threshold <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p $ </tex-math></inline-formula> to help determine regular behaviors. The experiment, which is conducted on a real-world dataset containing one-week vehicle passing records in Chongqing of China, validates the effectiveness and accuracy of the proposed model. Moreover, we analyze the mobility pattern of private car commuters, and some typical mobility patterns of them are successfully found.

Substructure in the stellar halo near the Sun

Context.Merger debris is expected to populate the stellar haloes of galaxies. In the case of the Milky Way, this debris should be apparent as clumps in a space defined by the orbital integrals of motion of the stars.Aims.Our aim is to develop a data-driven and statistics-based method for finding these clumps in integrals-of-motion space for nearby halo stars and to evaluate their significance robustly.Methods.We used data fromGaiaEDR3, extended with radial velocities from ground-based spectroscopic surveys, to construct a sample of halo stars within 2.5 kpc from the Sun. We applied a hierarchical clustering method that makes exhaustive use of the single linkage algorithm in three-dimensional space defined by the commonly used integrals of motion energyE, together with two components of the angular momentum,LzandL⊥. To evaluate the statistical significance of the clusters, we compared the density within an ellipsoidal region centred on the cluster to that of random sets with similar global dynamical properties. By selecting the signal at the location of their maximum statistical significance in the hierarchical tree, we extracted a set of significant unique clusters. By describing these clusters with ellipsoids, we estimated the proximity of a star to the cluster centre using the Mahalanobis distance. Additionally, we applied the HDBSCAN clustering algorithm in velocity space to each cluster to extract subgroups representing debris with different orbital phases.Results.Our procedure identifies 67 highly significant clusters (&gt; 3σ), containing 12% of the sources in our halo set, and 232 subgroups or individual streams in velocity space. In total, 13.8% of the stars in our data set can be confidently associated with a significant cluster based on their Mahalanobis distance. Inspection of the hierarchical tree describing our data set reveals a complex web of relations between the significant clusters, suggesting that they can be tentatively grouped into at least six main large structures, many of which can be associated with previously identified halo substructures, and a number of independent substructures. This preliminary conclusion is further explored in a companion paper, in which we also characterise the substructures in terms of their stellar populations.Conclusions.Our method allows us to systematically detect kinematic substructures in the Galactic stellar halo with a data-driven and interpretable algorithm. The list of the clusters and the associated star catalogue are provided in two tables available at the CDS.

Comparison of Hierarchical Clustering, K-Means, K-Medoids, and Fuzzy C-Means Methods in Grouping Provinces in Indonesia according to the Special Index for Handling Stunting

Stunting has been widely known as the highest case of malnutrition suffered by toddlers in the world and has a bad impact on children's future. In 2018, Indonesia was ranked the 31st highest stunting in the world and ranked 4th in Southeast Asia. About 30.8% (roughly 3 out of 10) of children under 5 years suffer from stunting in Indonesia. To support the government policy making in handling stunting, it is undoubtedly necessary to classify the levels of stunting handling in regions in Indonesia. In this work, the hierarchical agglomerative and non-hierarchical clustering is compared and evaluated to perform clustering on stunting data. The agglomerative hierarchical cluster uses Single Linkage, Average Linkage, Complete Linkage, and Ward Method, while the non-hierarchical cluster uses K-Means, K-Medoids (PAM) Clustering, and Fuzzy C-Means. This study uses data from 12 IKPS indicators in 34 provinces in Indonesia in 2018. Based on the results of the evaluation using the Connectivity Coefficient, Dunn Index, Silhouette Coefficient, Davies Bouldin Index, Xie &amp; Beni Index, and Calinski-Harabasz Index, the results show that the Average Linkage is the best cluster method with the optimal number of clusters is four clusters. The first cluster is a cluster with a good level of stunting management which consists of 28 provinces. The second cluster consists of only one province, DI Yogyakarta with a very good level of stunting handling. The third cluster consists of four provinces with poor stunting handling rates. Finally, the last cluster consisting of one province, Papua, has a very poor level of stunting handling.

Investigating maritime accidents that involve dangerous goods using hierarchical clustering

Although the important progress in terms of safety and technological advances, maritime accidents remain a critical issue in merchant shipping. A high number of accidents continue to occur every year, with negative consequences both in economic and environmental terms (with often disastrous and lasting environmental impacts for marine ecosystems) and in the loss of human life. Understanding the maritime accidents phenomenon is expedient to giving shipping practitioners a focus for tailored interventions aimed at enhancing maritime safety. Using hierarchical clustering methods, this paper analyses historical data relating to maritime accidents to highlight the potential causal relationships that can describe homogeneous groups of accidents. The study explores a database consisting of 1,079 marine accidents that occurred worldwide in the 2009-2019 decade. Accident data is taken from the International Maritime Organization (IMO) database. After illustrating a description of the data set, a non-supervised hierarchical clustering analysis is applied to identify accident patterns, thus helping to better describe the phenomenon and identify potential causal relations that repeat in various accidents. A significant distinction emerges between the accidents that occur for technical reasons and those where human factors (stress, fatigue, situation awareness, decision-making, communication, etc.) play a prevalent role. Afterwards, the clustering analysis is applied to a sub-set of accidents (153 accidents) involving ships carrying dangerous goods (gases, oils, explosives, etc.). The results of the analysis point out the role of the human factor as the prevalent (or contributing) cause of the marine accidents related to work operations. Conversely, fires and explosions, which are by far the most frequent accidents involving ships carrying dangerous goods, are mainly caused by technical problems.

Understanding the COVID-19 pandemic prevalence in Africa through optimal feature selection and clustering: evidence from a statistical perspective.

The COVID-19 pandemic, which outbroke in Wuhan (China) in December 2019, severely hit almost all sectors of activity in the world as a consequence of the restrictive measures imposed. Two years later, Africa still emerges as the least affected continent by the pandemic. This study analyzed COVID-19 prevalence across African countries through country-level variables prior to clustering. Using Spearman-rank correlation, multicollinearity analysis and univariate filtering, 9 country-level variables were identified from an initial set of 34 variables. These variables relate to socioeconomic status, population structure, healthcare system and environment and the climatic setting. A clustering of the 54 African countries is further carried out through the use of agglomerative hierarchical clustering (AHC) method, which generated 3 distinctive clusters. Cluster 1 (11 countries) is the most affected by COVID-19 (median of 63,508.6 confirmed cases and 946.5 deaths per million) and is composed of countries with the highest socioeconomic status. Cluster 2 (27 countries) is the least affected (median of 4473.7 confirmed cases and 81.2 deaths per million), and mainly features countries with the least socioeconomic features and international exposure. Cluster 3 (16 countries) is intermediate in terms of COVID-19 prevalence (median of 2569.3 confirmed cases and 35.7 deaths per million) and features countries the least urbanized and geographically close to the equator, with intermediate international exposure and socioeconomic features. These findings shed light on the main features of COVID-19 prevalence in Africa and might help refine effectively coping management strategies of the ongoing pandemic.Supplementary InformationThe online version contains supplementary material available at 10.1007/s10668-022-02646-3.

PEMETAAN KABUPATEN KOTA DI PROVINSI JAWA TIMUR BERDASARKAN TINGKAT KASUS PENYAKIT MENGGUNAKAN PENDEKATAN AGGLOMERATIF HIERARCHICAL CLUSTERING

Health is a human right that is one of the indicators of welfare that must felt by all people following the ideals of the Indonesian people to realize a high degree of health. The high level of public health is one of the base capital for realization of the implementation of national development. One indicator of policy target evaluation in the health sector is the availability of adequate public health facilities and services. In addition, the level of the number of people suffering from disease complaints needs to be the attention of the government and the community to become the material for evaluating policy targets in the health sector. Based on data on the percentage of the population who have health complaints in 2020 obtained from the BPS of the Republic of Indonesia, East Java Province is ranked 7th, which is 32.8%. While the data in 2021 East Java Province is ranked 8th at 28.55%. This figure is quite high compared to other provinces so the East Java Provincial government needs to pay attention to reducing cases of disease complaints. Therefore, this study aims to analyze and map regencies and cities in East Java Province based on the distribution of the characteristics of the number of disease cases in each regency and city using the Agglomerative Hierarchical Clustering method. Based on the value of the cophenetic correlation coefficient, the average linkage method is the best method with the optimal number of clusters formed 4 clusters based on the cluster validation test using the connectivity validity index test, Dunn index, and silhouette. The results of this study can be used as a reference for the government of East Java Province in taking policies to improve national development planning in the health sector to increase the high level of public health

A dynamic large-scale multiple attribute group decision-making method with probabilistic linguistic term sets based on trust relationship and opinion correlation

Dynamic large-scale multiple attribute group decision making (DLMAGDM) is ubiquitous in many areas of the real world. It is composed of large numbers of decision makers, several continuous periods, alternative set and attribute set changed with time. Given the characteristics implicited in decision-making elements and the advantages of probabilistic linguistic term sets (PLTSs) in modelling uncertainty and complexity of decision makers’ subjective opinions, this paper constructs a probabilistic linguistic DLMAGDM method. First of all, a dynamic weight determination model based on trust relationships and evidential conflicts between decision makers is proposed to obtain current dynamic weights of decision makers. Then, a comprehensive hierarchical clustering method that divides large numbers of decision makers into several subgroups is constructed based on three clustering constrains. Moreover, some probabilistic linguistic extended evidential power aggregation operators are proposed to aggregate PLTSs. These operators can appropriately handle the extreme PLTSs and fully consider the role of incomplete probabilistic distributions in PLTSs. In addition, a dynamic decision-making method based on PROMETHEE is developed to determine the final priority order of alternatives according to preferences between alternatives over several periods. Lastly, a case study for supply chain finance risk assessment for several firms in Chinese household appliance industry is utilized to illustrate the practicality and effectiveness of the probabilistic linguistic DLMAGDM method. Furthermore, the comparative analysis with some other existing methods and the sensitivity analysis are made to verify its advantages.

Biomarker characterization of clinical subtypes of Parkinson Disease

The biological underpinnings of the PD clusters remain unknown as the existing PD clusters lacks biomarker characterization. We try to identify clinical subtypes of Parkinson Disease (PD) in an Asian cohort and characterize them by comparing clinical assessments, genetic status and blood biochemical markers. A total of 206 PD patients were included from a multi-centre Asian cohort. Hierarchical clustering was performed to generate PD subtypes. Clinical and biological characterization of the subtypes were performed by comparing clinical assessments, allelic distributions of Asian related PD gene (SNCA, LRRK2, Park16, ITPKB, SV2C) and blood biochemical markers. Hierarchical clustering method identified three clusters: cluster A (severe subtype in motor, non-motor and cognitive domains), cluster B (intermediate subtype with cognitive impairment and mild non-motor symptoms) and cluster C (mild subtype and young age of onset). The three clusters had significantly different allele frequencies in two SNPs (Park16 rs6679073 A allele carriers in cluster A B C: 67%, 74%, 89%, p = 0.015; SV2C rs246814 T allele distribution: 7%, 12%, 25%, p = 0.026). Serum homocysteine (Hcy) and C-reactive protein (CRP) levels were also significantly different among three clusters (Mean levels of Hcy and CRP among cluster A B C were: 19.4 ± 4.2, 18.4 ± 5.7, 15.6 ± 5.6, adjusted p = 0.005; 2.5 ± 5.0, 1.5 ± 2.4, 0.9 ± 2.1, adjusted p < 0.0001, respectively). Of the 3 subtypes identified amongst early PD patients, the severe subtype was associated with significantly lower frequency of Park16 and SV2C alleles and higher levels of Hcy and CRP. These biomarkers may be useful to stratify PD subtypes and identify more severe subtypes.

Layout Method of Met Mast Based on Macro Zoning and Micro Quantitative Siting in a Wind Farm

In order to promote the wind monitoring accuracy and provide a quantitative planning method for met mast layout in practical projects, this paper proposes a two-stage layout method for met mast based on discrete particle swarm optimization (DPSO) zoning and micro quantitative siting. Firstly, according to the wind turbines layout, rotational empirical orthogonal function and hierarchical clustering methods are used to preliminarily determine zoning number. Considering the geographical proximity of wind turbines and the correlation of wind speed, an optimal macro zoning model of wind farm based on improved DPSO is established. Then, combined with the grid screening method and optimal layout evaluation index, a micro quantitative siting method of met mast is proposed. Finally, the rationality and efficiency of macro zoning method based on improved DPSO, as well as the objectivity and standardization of micro quantitative siting, are verified by an actual wind farm.

Literature-based drug-drug similarity for drug repurposing: impact of Medical Subject Headings term refinement and hierarchical clustering.

Background: We describe herein, an improved procedure for drug repurposing based on refined Medical Subject Headings (MeSH) terms and hierarchical clustering method. Materials & methods: In the present study, we have employed MeSH data from MEDLINE (2019), 1669 USFDA approved drugs from OpenFDA and a refined set of MeSH terms. Refinement of MeSH terms was performed to include terms related to mechanistic information of drugs and diseases. Results and Conclusions: In-depth analysis of the results obtained, demonstrated greater efficiency of the proposed approach, based on refined MeSH terms and hierarchical clustering, in terms of number of selected drug candidates for repurposing. Further, analysis of misclustering and size of noise clusters suggest that the proposed approach is reliable and can be employed in drug repurposing.

Wind variability over a large lake with complex topography: Lake of the Woods

Lake of the Woods spans two geological regions: the Precambrian Shield and the Glacial Lake Agassiz Lakebed. The lake watershed’s complex topography influences wind speeds and directions. We study wind variability over Lake of the Woods using reanalysis data and wind observations at seven sites. A wind variability index is proposed as a measure of lake-wide spread in wind direction. Our results show that the directional variability depends on wind speeds, with higher variability at lower speeds. The wind variability also changes throughout the year; it is more pronounced in spring (Apr-Jun) and fall (Sep-Oct) and less pronounced during winter (Dec-Jan). Observed wind directions are 60% more variable than reanalysis wind, indicating a need for higher-resolution reanalysis to resolve local wind effects due to topographic changes. Based on the statistical hierarchical clustering method, our analysis shows that geographically variable wind can be grouped into three regions to represent the predominant wind directions accurately. Clustering results also suggest spatial coverage gaps in the existing meteorological observation network. We reassess the existing network and propose a new meteorological observation site in the southeast basin near Morson, Ontario. Our study is of significance for water quality management because the wind is the key driver of water circulation, transport, and mixing in the lake, and accurate wind measurements are necessary to better constrain the underpinning physical, biological, and biochemical processes in mechanistic ecological models.

Innovative Energy Technologies in Road Transport in Selected EU Countries

The primary aim of this study was to assess and classify selected EU countries to groups differing in terms of the degree of implementation of innovative energy technologies to alleviate adverse externalities in road transport. This aim was realised using three groups of research methods: collection of empirical data, data processing and presentation of study outcomes. When collecting the research material, the authors used the method of critical literature review and the documentation method. The research material was processed using the agglomerative clustering technique, which was one of the hierarchical clustering methods. The distance between objects (here, selected EU countries) was determined based on the Euclidean distance. The outcome of this analysis was a dendrogram, which constitutes a graphical interpretation of obtained results. The study was conducted on 21 EU countries. The analyses covered the years 2013–2019. The sources of materials included literature on the subject and the Eurostat data. The problem of innovative energy technologies in road transport is presently of considerable importance. This results from the current situation related to human activity. As a result of the conducted cluster analysis, groups were distinguished based on differences in the use of innovative energy technologies alleviating negative externalities generated by road transport. The first group comprised Sweden, the Netherlands and Finland. Compared to the other groups, this group was distinguished by the highest values of four indexes, i.e., the share of renewable energy sources used in transport in 2019, the share in the market of electric passenger vehicles in 2019, the share in the market of electric lorries in 2019, as well as the share in the market of hybrid automobiles in 2019. Countries which participated the least in the elimination of negative externalities generated by road transport included Romania, Hungary, Greece, Poland, Latvia and Estonia.

Identification of Sports Athletes Psychological Stress Based on K-Means Optimized Hierarchical Clustering.

In order to solve the problem, the psychological identification of athletes in professional competition pressure is difficult. This paper first analyzes the sources of athletes' psychological pressure based on the hierarchical clustering method, and then divides the weights of the sources of psychological pressure, quantificationally scores them and constructs an identification model of athletes' psychological pressure. Then, the clustering process is optimized based on the K-Means algorithm, and its effectiveness is verified. Finally, the psychological stress of 10 players in a football club was analyzed. The results show that the model effectively and reasonably reflects the influence of pressure sources on the athletes' competitive state during the competition, which provides a basis for the decision-making of relief about athletes' stress.

Unsupervised clustering of SARS-CoV-2 using deep convolutional autoencoder

SARS-CoV-2’s population structure might have a substantial impact on public health management and diagnostics if it can be identified. It is critical to rapidly monitor and characterize their lineages circulating globally for a more accurate diagnosis, improved care, and faster treatment. For a clearer picture of the SARS-CoV-2 population structure, clustering the sequencing data is essential. Here, deep clustering techniques were used to automatically group 29,017 different strains of SARS-CoV-2 into clusters. We aim to identify the main clusters of SARS-CoV-2 population structure based on convolutional autoencoder (CAE) trained with numerical feature vectors mapped from coronavirus Spike peptide sequences. Our clustering findings revealed that there are six large SARS-CoV-2 population clusters (C1, C2, C3, C4, C5, C6). These clusters contained 43 unique lineages in which the 29,017 publicly accessible strains were dispersed. In all the resulting six clusters, the genetic distances within the same cluster (intra-cluster distances) are less than the distances between inter-clusters (P-value 0.0019, Wilcoxon rank-sum test). This indicates substantial evidence of a connection between the cluster’s lineages. Furthermore, comparisons of the K-means and hierarchical clustering methods have been examined against the proposed deep learning clustering method. The intra-cluster genetic distances of the proposed method were smaller than those of K-means alone and hierarchical clustering methods. We used T-distributed stochastic-neighbor embedding (t-SNE) to show the outcomes of the deep learning clustering. The strains were isolated correctly between clusters in the t-SNE plot. Our results showed that the (C5) cluster exclusively includes Gamma lineage (P.1) only, suggesting that strains of P.1 in C5 are more diversified than those in the other clusters. Our study indicates that the genetic similarity between strains in the same cluster enables a better understanding of the major features of the unknown population lineages when compared to some of the more prevalent viral isolates. This information helps researchers figure out how the virus changed over time and spread to people all over the world.

Education signaling, effort investments, and the market's expectations: Theory and experiment on China's higher education expansion

China's higher education expansion has led to significant changes in younger generations' educational investments and labor market outcomes, and this trend is expected to continue due to the recent post-graduate education expansion in response to economic challenges posed by the COVID-19 pandemic. This paper investigates the impact of higher education expansion on labor market participants' choices, beliefs, and learning effects through an extended education signaling model, and uncovers the behavioral patterns in response to this policy change using a laboratory experiment. We find that subjects playing the role of workers generally follow a threshold strategy, and the changes in their effort investments and received wages after the enrollment expansion are consistent with the theoretical prediction. Using a hierarchical clustering method, we estimate different types of empirical strategies adopted by the subjects. In the low-enrollment treatment, the three types of empirical strategies are more distinct, and there is a strong tendency for effort over-investment by low-ability workers and under-investment by high-ability workers. In the high-enrollment treatment, the distinction between the estimated strategy types becomes much smaller. An analysis using elicited beliefs suggests that effort over-investment stems from workers' inconsistent beliefs regarding firms' wage offers — this inconsistency persists even in the last few periods of the game. Our findings provide a belief-based explanation for the discussion on over-education and are of great policy relevance.