Agglomerative Hierarchical Method Research Articles

Voting Simulation based Agglomerative Hierarchical Method for Network Community Detection

Community detection has been paid much attention in many fields in recent years, and a great deal of community-detection methods have been proposed. But the time consumption of some of them is heavy, limiting them from being applied to large-scale networks. On the contrary, there exist some lower-time-complexity methods. But most of them are non-deterministic, meaning that running the same method many times may yield different results from the same network, which reduces their practical utility greatly in real-world applications. To solve these problems, we propose a community-detection method in this paper, which takes both the quality of the results and the efficiency of the detecting procedure into account. Moreover, it is a deterministic method which can extract definite community structures from networks. The proposed method is inspired by the voting behaviours in election activities in the social society, in which we first simulate the voting procedure on the network. Every vertex votes for the nominated candidates following the proposed voting principles, densely connected groups of vertices can quickly reach a consensus on their candidates. At the end of this procedure, candidates and their own voters form a group of clusters. Then, we take the clusters as initial communities, and agglomerate some of them into larger ones with high efficiency to obtain the resulting community structures. We conducted extensive experiments on some artificial networks and real-world networks, the experimental results show that our proposed method can efficiently extract high-quality community structures from networks, and outperform the comparison algorithms significantly.

Short-Term Wind Power Forecasting Based on Clustering Pre-Calculated CFD Method

To meet the increasing wind power forecasting (WPF) demands of newly built wind farms without historical data, physical WPF methods are widely used. The computational fluid dynamics (CFD) pre-calculated flow fields (CPFF)-based WPF is a promising physical approach, which can balance well the competing demands of computational efficiency and accuracy. To enhance its adaptability for wind farms in complex terrain, a WPF method combining wind turbine clustering with CPFF is first proposed where the wind turbines in the wind farm are clustered and a forecasting is undertaken for each cluster. K-means, hierarchical agglomerative and spectral analysis methods are used to establish the wind turbine clustering models. The Silhouette Coefficient, Calinski-Harabaz index and within-between index are proposed as criteria to evaluate the effectiveness of the established clustering models. Based on different clustering methods and schemes, various clustering databases are built for clustering pre-calculated CFD (CPCC)-based short-term WPF. For the wind farm case studied, clustering evaluation criteria show that hierarchical agglomerative clustering has reasonable results, spectral clustering is better and K-means gives the best performance. The WPF results produced by different clustering databases also prove the effectiveness of the three evaluation criteria in turn. The newly developed CPCC model has a much higher WPF accuracy than the CPFF model without using clustering techniques, both on temporal and spatial scales. The research provides supports for both the development and improvement of short-term physical WPF systems.

Identifying regional poverty types in Slovakia

Even today the poverty is an important negative social phenomenon with evident spatial dimension. Areas with high poverty concentration constitute poverty regions with plenty of common but also different or specific characteristics. Research into the specific poverty features in regions lags behind that of the common characteristics and character of poverty. The aim is to show one of ways to carry out typification of poverty regions (on example of Slovakia). A universal conceptual and methodological frame applicable to any country is presented. Regional poverty types were sorted out by application of the hierarchical agglomerative methods of cluster analysis (Ward’s method) with nine indicators capturing the generally applicable dimensions, which determine poverty and profile of poverty regions. Results confirmed the presumption that the different economic, social, demographic, cultural and other different conditions and population structure in regions may determine the character of poverty. Obtained knowledge points not only to the particular population groups, which require attention but also to the regional poverty types that should be identified for the efficient fight against poverty in these regions.

Assessing the spatial variation of functional diversity estimates based on dendrograms in phytoplankton communities

ABSTRACT Variation in phytoplankton functional diversity is partioned and mapped using several indices and linkage methods based on dendrograms. The relationships between diversity metrics and major environmental predictors, including zooplankton density, were assessed in 29 phytoplankton communities of floodplain lakes distributed along the Middle Araguaia River in central Brazil. The dendrogram-based functional diversity indices were Functional Group Richness, Functional Diversity, Mean Pairwise Distance and Mean Nearest Taxon Distance, whereas seven different hierarchical agglomerative linkage methods we used. The performance of indices were compared using ANOVA and their spatial variation in response to major environmental predictors evaluated. The results indicate that variation in functional diversity values is primarily a product of the type of index chosen. This variation was statistically significant in 90 % of the floodplain lakes studied; however, a spatial pattern of variation in index values along the river was not detected. Furthermore, environmental constraints, including zooplankton density, were weak predictors of functional diversity indices. Therefore, the mathematical characteristics of indices are of primary importance in explaining variation among functional diversity values.

The multiple outliers detection using agglomerative hierarchical methods in circular regression model

Two agglomerative hierarchical clustering algorithms for identifying multiple outliers in circular regression model have been developed in this study. The agglomerative hierarchical clustering algorithm starts with every single data in a single cluster and it continues to merge with the closest pair of clusters according to some similarity criterion until all the data are grouped in one cluster. The single-linkage method is one of the simplest agglomerative hierarchical methods that is commonly used to detect outlier. In this study, we compared the performance of single-linkage method with another agglomerative hierarchical method, namely average linkage for detecting outlier in circular regression model. The performances of both methods were examined via simulation studies by measuring their “success” probability, masking effect, and swamping effect with different number of sample sizes and level of contaminations. The results show that the single-linkage method performs very well in detecting the multiple outliers with lower masking and swamping effects.

Hierarchical clustering for histogram data

Clustering methods for classical data are well established, though the associated algorithms primarily focus on partitioning methods and agglomerative hierarchical methods. With the advent of massively large data sets, too large to be analyzed by traditional techniques, new paradigms are needed. Symbolic data methods form one solution to this problem. While symbolic data can be important and arise naturally in their own right, they are particularly relevant when faced with data that emerged from aggregation of (larger) data sets. One format is when the data are histogram‐valued in ℝp, instead of points in ℝp as in classical data. This paper looks at the problem of constructing hierarchies using a divisive polythetic algorithm based on dissimilarity measures derived for histogram observations. WIREs Comput Stat 2017, 9:e1405. doi: 10.1002/wics.1405This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification

Night Sky Brightness Assessment in Nigeria Using Environmetric and GIS Technique

High anthropogenic activities are rapidly increasing phenomenon and it is assumed to have global implications. In Nigeria, high anthropogenic activities rapidly increase above the standard of the threshold values especially the lighting sources. This increase due to the vastly growing of industries, residential and commercial uses and other sources such as street lighting in urban and semi-urban areas, which can make the night sky brightness in the area above the threshold set for polluted status. The study measures the night sky brightness at the most densely populated urban centers of Nigeria; to estimate and quantify the level of the night sky brightness from the sites nearby the cities of the planned research study. The research monitored the zenith sky brightness from November 2015 to March 2016 using Sky Quality Meter (SQM). However, typical values ranging from 20.14 to 22.00Mag.sqm /arc sec.2 were measured at different sites of the study area. This data recorded in the field was analyzed using Agglomerative Hierarchical Method via Ward’s Methods to cluster our data according to the pollution status of the areas. Result showed three clusters in which class 1 has low pollution; class 2 is moderate while class 3 has the high pollution status. The sites classified in class 3 are more polluted due to the high use of artificial light. However, geographical information system (GIS) software was employed to confirm the results obtained from environ-metric technique and concluded that this result is confirmed to be the same. Hence, it illustrated that sites in cluster 1 have an excellent dark location that can be used to build optical observatory stations and other astronomical observations due to their dark sky.

Análise exploratória de simulações sociais computacionais por meio de estatística multivariada e mapas auto-organizáveis

Este trabalho teve como objetivo a aplicação da análise de componentes principais, da análise de agrupamentos pelo método hierárquico aglomerativo e da rede neural do tipo Mapa Auto-Organizável de Kohonen na análise exploratória de resultados de simulações sociais computacionais do sistema socioterritorial “Território Rural Sul Sergipano”. A metodologia basea-se na Sociologia da Ação Organizada e no método Soclab. As análises estatísticas mostraram que as simulações do sistema socioterritorial geraram resultados que confirmam a tendência do sistema à estabilidade. No entanto, observou-se, por meio da rede neural, que há diferenças, ainda que pequenas, entre as situações que levam o sistema à estabilidade.

An Effective Approach for Clustering InhA Molecular Dynamics Trajectory Using Substrate-Binding Cavity Features.

Protein receptor conformations, obtained from molecular dynamics (MD) simulations, have become a promising treatment of its explicit flexibility in molecular docking experiments applied to drug discovery and development. However, incorporating the entire ensemble of MD conformations in docking experiments to screen large candidate compound libraries is currently an unfeasible task. Clustering algorithms have been widely used as a means to reduce such ensembles to a manageable size. Most studies investigate different algorithms using pairwise Root-Mean Square Deviation (RMSD) values for all, or part of the MD conformations. Nevertheless, the RMSD only may not be the most appropriate gauge to cluster conformations when the target receptor has a plastic active site, since they are influenced by changes that occur on other parts of the structure. Hence, we have applied two partitioning methods (k-means and k-medoids) and four agglomerative hierarchical methods (Complete linkage, Ward’s, Unweighted Pair Group Method and Weighted Pair Group Method) to analyze and compare the quality of partitions between a data set composed of properties from an enzyme receptor substrate-binding cavity and two data sets created using different RMSD approaches. Ensembles of representative MD conformations were generated by selecting a medoid of each group from all partitions analyzed. We investigated the performance of our new method for evaluating binding conformation of drug candidates to the InhA enzyme, which were performed by cross-docking experiments between a 20 ns MD trajectory and 20 different ligands. Statistical analyses showed that the novel ensemble, which is represented by only 0.48% of the MD conformations, was able to reproduce 75% of all dynamic behaviors within the binding cavity for the docking experiments performed. Moreover, this new approach not only outperforms the other two RMSD-clustering solutions, but it also shows to be a promising strategy to distill biologically relevant information from MD trajectories, especially for docking purposes.

Macrozoobenthos communities in the area of marine pipeline route at Piltun-Astokhskoye oil field (shelf of northeastern Sakhalin) and their long-term variability

Results of macrozoobenthos surveys (in total 86 stations, 432 sediment samples) in the area of marine pipeline route at Piltun-Astokhskoye oil field in July of 2007-2010 are presented. The sediment samples were collected by van Veen sampler (1-3 samples per site from the area 0.05 m 2 , 0.11 m 2 , or 0.25 m 2 ) at the depth 0-45 m. The samples were washed by sea water through the sieve with mesh 1 mm, and the residues including macrobenthos were preserved by 4 % buffered formaldehyde for later processing with total identification of macrozoobenthos (species composition, abundance and biomass) in the laboratories of Far-Eastern Research Hydrometeorological Institute and Institute of Marine Biology in Vladivostok, following to standard techniques. The data were analysed using the PRIMER software including multidimensional cluster analysis of samples by hierarchical agglomerative method of group-average linking based on Bray-Curtis similarity of the macrozoobenthos species composition by biomass. In total, 272 species of macrozoobenthos are identified at the surveyed area, with the highest species number for amphipods (86 species) and polychaetes (72 species). Mean total abundance of macrozoobenthos is 26556.4 ± 2660.9 ind/m 2 , its mean total biomass is 669.7 ± 40.3 g/m 2 . Cumaceans and amphipod crustaceans are the most numerous groups; sea-urchins, bivalves and cumaceans have the highest biomass. Five benthic communities are defined by cluster analysis. There is concluded that the bottom sediments and depth are the main factors determining the characteristics and distribution patterns of benthic assemblages at present time.

Symptoms and Problem Clusters in Cancer and Non-Cancer Patients in Specialized Palliative Care—Is There a Difference?

ContextIn clinical practice, some symptoms and problems frequently occur in combination, which may have consequences for symptom management. ObjectivesFacing a growing number of non-cancer patients in palliative care, this study aimed to differentiate symptom clusters in the non-cancer population from those in cancer patients. MethodsInpatient data from the German Hospice and Palliative Care Evaluation between 2007 and 2011 were used for a cluster analysis of a 16-item symptom and problem checklist. An agglomerative hierarchical method was chosen. Coefficients from distance matrix ranging between 0 and 1 were calculated to indicate the interrelationship of clustered symptoms. ResultsThe analysis identified five clusters in cancer patients: 1) nausea and vomiting (d = 0.000); 2) anxiety, tension, and feeling depressed (d = 0.125); 3) wound care and disorientation/confusion (d = 0.229); 4) organization of care and overburdening of family (d = 0.202); and 5) weakness, tiredness, need for assistance with activities of daily living, and loss of appetite (d = 0.207). Five comparable clusters were identified in non-cancer patients: 1) nausea and vomiting (d = 0.000); 2) anxiety, tension, and feeling depressed (d = 0.166); 3) organization of care and overburdening of family (d = 0.187); 4) weakness and need for assistance with activities of daily living (d = 0.139); and 5) tiredness and loss of appetite (d = 0.182). ConclusionAs symptom clusters do not significantly differ between cancer and non-cancer patients, specific frequent symptoms in non-cancer patients should be assessed. Identification of symptom clusters may help to target therapies and focus the use of medications to improve patients' quality of life.

A cophenetic correlation coefficient for Tocher's method

The objective of this work was to propose a way of using the Tocher's method of clustering to obtain a matrix similar to the cophenetic one obtained for hierarchical methods, which would allow the calculation of a cophenetic correlation. To illustrate the obtention of the proposed cophenetic matrix, we used two dissimilarity matrices - one obtained with the generalized squared Mahalanobis distance and the other with the Euclidean distance - between 17 garlic cultivars, based on six morphological characters. Basically, the proposal for obtaining the cophenetic matrix was to use the average distances within and between clusters, after performing the clustering. A function in R language was proposed to compute the cophenetic matrix for Tocher's method. The empirical distribution of this correlation coefficient was briefly studied. For both dissimilarity measures, the values of cophenetic correlation obtained for the Tocher's method were higher than those obtained with the hierarchical methods (Ward's algorithm and average linkage - UPGMA). Comparisons between the clustering made with the agglomerative hierarchical methods and with the Tocher's method can be performed using a criterion in common: the correlation between matrices of original and cophenetic distances.

Selection of key ambient particulate variables for epidemiological studies — Applying cluster and heatmap analyses as tools for data reduction

The success of epidemiological studies depends on the use of appropriate exposure variables. The purpose of this study is to extract a relatively small selection of variables characterizing ambient particulate matter from a large measurement data set. The original data set comprised a total of 96 particulate matter variables that have been continuously measured since 2004 at an urban background aerosol monitoring site in the city of Augsburg, Germany. Many of the original variables were derived from measured particle size distribution (PSD) across the particle diameter range 3nm to 10μm, including size-segregated particle number concentration, particle length concentration, particle surface concentration and particle mass concentration. The data set was complemented by integral aerosol variables. These variables were measured by independent instruments, including black carbon, sulfate, particle active surface concentration and particle length concentration. It is obvious that such a large number of measured variables cannot be used in health effect analyses simultaneously. The aim of this study is a pre-screening and a selection of the key variables that will be used as input in forthcoming epidemiological studies.In this study, we present two methods of parameter selection and apply them to data from a two-year period from 2007 to 2008. We used the agglomerative hierarchical cluster method to find groups of similar variables. In total, we selected 15 key variables from 9 clusters which are recommended for epidemiological analyses. We also applied a two-dimensional visualization technique called “heatmap” analysis to the Spearman correlation matrix. 12 key variables were selected using this method. Moreover, the positive matrix factorization (PMF) method was applied to the PSD data to characterize the possible particle sources. Correlations between the variables and PMF factors were used to interpret the meaning of the cluster and the heatmap analyses.

Assessment of metals in fourteen species of vegetables and crops cultivated in a suburban area using multivariate analyses

Concentrations of seven metals (Cd, Cr, Mn, Ni, Pb, V, and Zn) were determined in 14 plant species including vegetables and crops from an agricultural area located at the Southern part of the Iranian capital, near the Tehran oil refinery. Different statistical methods, especially multivariate ones, were employed for the analysis of the results. Based on the results of agglomerative hierarchical method for clustering of the grid cells, some similarities between the dendrograms of Pb, Ni, and V and, furthermore, between Mn and Cr were observed that could be attributed to their similar distribution sources in the study area. The principal component analysis showed that with respect to accumulation of all the metals in the plant species, two main groups could be identified. The highest mean concentrations of all metals (except for Pb) were found in onions. In all plants the mean concentrations of Cd and Pb were considerably higher than limits stated in international guidelines for human consumption. The levels of the metals in onions exceeded these limits. Moreover, the Cr concentrations in mint and basil and the Zn concentrations in tarragon and parsley were higher than the established limits.

Ambientalismo e concepções de RESEX, extrativismo e conhecimento no ICMBIO na Amazônia Legal

O trabalho discute o posicionamento de técnicos de nível superior do Instituto Chico Mendes de Conservação da Biodiversidade ICMBIO , em exercício na Amazônia Brasileira, em relação ao ambientalismo e identifica possíveis relações entre os posicionamentos e visões do conhecimento, concepção de Reserva Extrativista RESEX e de extrativismo. Consoante a base teórica discutida, foram adotadas as seguintes tipologias para as categorias estudadas: correntes ou vertentes ambientalistas: preservacionista, Sustentabilista e Socioambientalista; visões do conhecimento: anarquista, dialógica ou saber ambiental ; interdisciplinar; e, científica pura; visões de RESEX: RESEX como Unidade de Conservação UC de uso sustentável, RESEX servindo à reforma agrária e RESEX oportunista; visões sobre extrativismo: extrativismo legítimo, extrativismo falido e extrativismo oportunista. O método utilizado foi o comparativo com partições cruzadas, verificando similaridades e dissimilaridades de grupos de respondentes. Para levantamento dos posicionamentos dos respondentes foi disponibilizado a todos os técnicos um questionário com afirmações representativas das diferentes visões, utilizando escala do tipo Likert. Foi realizada a classificação dos técnicos em categorias, utilizando análise de agrupamento pelo método hierárquico aglomerativo, tendo como parâmetro a proximidade de posicionamento. Constatou-se que não há visão única a respeito do ambientalismo no Instituto Chico Mendes na Amazônia Legal e não existem linhas ambientalistas puras, mas compostas por concepções de várias vertentes. As visões sobre RESEXs e extrativismo estão relacionadas com concepções sociaombientalistas, reforçando a defesa de que este tipo de Unidade de Conservação faz parte dessa linha de pensamento.

Clustering of Distributions: A Case of Patent Citations

Often the data units are described with discrete distributions (work described with citation distribution over time, population pyramid described as age-sex distribution etc.).When the set of such units is very large, appropriate clustering methods can reveal the typical patterns hidden in the data. In this paper we present an adapted leaders method combined with a compatible adapted agglomerative hierarchical method that are based on relative error measure between a unit and the corresponding cluster representative–leader. The proposed approach is illustrated on citation distributions derived from the data set of US patents from 1980 to 1999. These new methods were developed because clustering of units, described with distributions, with classical k-means method reveals patterns with single high peaks which correspond to a single year. These patterns prevail over other distribution shapes also present in the data. Compared with centers in k-means method, clusters’ representatives obtained with the proposed new methods better detect typical distribution shapes of units. The obtained main cluster types for different sets of units show three main patterns: patents with early or late peak of importance to the community, and patents where the importance is slowly increasing throughout the time period.

Three statistical methods for sex determination in extant and fossil caprines: assessment of the Rupicapra long bones

Sex determination of fossil remains in archeopaleontological contexts is a necessary step in order to gain insights in archaeological and paleobiological parameters such as prey selection and sexual dimorphism. Species with higher sexual dimorphism are more reliably sexed while the sex determination of less dimorphic animals is not attempted. In this study, we compared the accuracy of three different methods in the sex determination of a modern known-sex sample of chamois ( Rupicapra pyrenaica parva) from the Cantabrian range (Northern Iberian Peninsula), which is a small-sized ungulate with low sexual dimorphism when compared to other animals in which sex determination has been attempted (i.e., Ursus spelaeus, Capra pyrenaica). The three methods applied were mixture analysis (MA), cluster analysis (CA) and principal component analysis (PCA). Within CA we applied three different analysis: one agglomerative hierarchical method (AH) and two optimal partitioning methods (k-means and robust k-means); and within PCA we applied normal and robust variants of the analysis. The performance of the CA and PCA methods was better than the MA. More specifically, we used the k-means analysis to the early Upper Pleistocene sample of chamois from Valdegoba cave (Burgos, Spain). The results show that chamois males double the number of females in this site, while the pattern in modern living chamois populations is the opposite. Moreover, the proportion of adults represented in the Valdegoba chamois sample is the 80.25% of the minimum number of individuals (MNI), well above the ∼50% expected in modern living populations. However, it is difficult to ascertain if Neandertals, the main accumulation agents of the site, were selecting young male chamois adults, or if neither sex was preferentially targeted, and males were hunted more frequently due to their solitary nature. Finally, the size of the chamois males at the site was larger than that of modern chamois males, while females are equal in size to those of present day. This discrepancy implies larger sexual dimorphism for the Pleistocene chamois sample from Valdegoba.

Quantifying trends in indicator hydroecological variables for regime‐based groups of Canadian rivers

AbstractAs a key contribution to Canada's Ecosystem Status and Trends (ESTR) national assessment, the goal of our study was to utilize available flow data as a surrogate of habitat suitability for aquatic ecological communities, and examine temporal trends in hydroecological variables over the 1970–2005 period. Daily flow data were extracted from the Reference Hydrological Basin Network, and an agglomerative hierarchical classification method was used to identify homogenous regions with similar seasonality of the flow regime. Six regime regions were identified reflecting the timing of the annual peaks and low flows in addition to the patterns in the rising and falling limbs. For each of the gauging station sites, the magnitude, duration, timing, frequency, and rate of change of annual hydrological events were quantified through 32 ecologically important hydrological variables. Long‐term patterns in the hydroecological variables were quantified using the nonparametric Mann‐Kendall trend statistic. The results revealed more trends than would be expected to occur by chance for most variables. Clear regional trend patterns were observed within individual regime groups demonstrating the often differing response to environmental variability within the different regions. Results at the national scale were highly variable, but trends towards increased variability in river flows were observed with a predominant increasing trend in the number of flow reversals over the water year and decreasing trend in the annual low‐flow indices. The identified river regime regions offer an initial framework for scientific investigation of hydroecological patterns and an opportunity to move towards a more predictive approach to environmental flows assessment in sustainable resource management and planning. Copyright © 2011 John Wiley & Sons, Ltd.

Clustering large data sets described with discrete distributions and its application on TIMSS data set

AbstractSymbolic data analysis is based on special descriptions of data—symbolic objects. Such descriptions preserve more detailed information about the data than the standard representations with mean values. A special kind of symbolic object is also representation with distributions. In the clustering process this representation enables us to consider the variables of all types at the same time.We present two clustering methods based on the data descriptions with discrete distributions: the adapted leaders method and the adapted agglomerative hierarchical clustering Ward's method. Both methods are compatible—they can be viewed as two approaches for solving the same clustering optimization problem. In the obtained clustering, the leader is assigned to each cluster. The descriptions of the leaders offer simple interpretation of the clusters' characteristics. The leaders method enables us to efficiently solve clustering problems with a large number of units; while the agglomerative method is applied on the obtained leaders and enables us to decide upon the right number of clusters on the basis of the corresponding dendrogram.Both methods were successfully applied in analyses of different data sets. In this paper, an application on the Trends in International Mathematics and Science Study (TIMSS) data set is presented. The descriptions with distributions enable us to combine two data sets: answers of teachers and answers of their students, into one data set. The descriptions of the obtained clusters enable us to interpret the results in a more understandable way. © 2011 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 4: 199–215, 2011

Application of the differential equations method for identifying communities in sparse networks

In our previous work a new method of identifying communities in networks was presented. The method is based on a time evolution of the network according to a set of differential equations. It was applied to networks consisting of fully connected sub-networks. However, networks describing real systems are often sparse. Here the method is applied to sparse networks. The results are compared with those of an agglomerative hierarchical method based on modularity maximisation proposed by Newman and a divisive method proposed by Duch and Arenas based also on optimisation of the modularity. Obtained results show that the differential equation method usually works better than two remaining methods, allowing for more appropriate identification of the network structure.