Agglomerative Techniques Research Articles

Identifying Communities with Modularity Metric Using Louvain and Leiden Algorithms

Over the past 20 years, there has been a significant increase in publication in complex network analysis research, especially in community detection. Many methods were proposed to identify community structure. Each community identification algorithm has strengths and weaknesses due to the complexity of information. Among them, the optimisation methods are widely focused on. This paper focuses on an empirical study of two community detection algorithms based on agglomerative techniques using modularity metric: Louvain and Leiden. In this regard, the Louvain algorithm has been shown to produce a bad connection in the community and disconnected when executed iteratively. Therefore, the Leiden algorithm is designed to successively resolve the weaknesses. Performance comparisons between the two and their concept were summarised in detail, as well as the step-by-step learning process of the state-of-the-art algorithms. This study is important and beneficial to the future study of interdisciplinary data sciences of network analysis. First, it demonstrates that the Leiden method outperformed the Louvain algorithm in terms of modularity metric and running time. Second, the paper displays the use of these two algorithms on synthetic and real networks. The experiment was successful as it identified better performance, and future work is required to confirm and validate these findings.

Comparação de métodos de agrupamentos hierárquicos aglomerativos em indicadores de sustentabilidade em municípios do estado do Pará

Este estudo teve como objetivo comparar o desempenho dos métodos de agrupamento hierárquico aglomerativo utilizando um conjunto de dados composto por diversos indicadores de sustentabilidade referentes aos municípios do estado do Pará. Assim como, definir a quantidade de agrupamentos iniciais a serem constituídos pela utilização dos índices de validade. Para seleção dos indicadores, foi feito um check-list de estudos científicos de abrangência nacional, regional e local que abordam a temática da sustentabilidade. Posteriormente, foi realizado a padronização dos indicadores, devido às unidades e escalas de medidas diferentes, não interferindo no resultado e possuindo pesos semelhantes no cômputo do coeficiente de similaridade. A medida de dissimilaridade empregada foi a distância euclidiana, e para determinar o método de agrupamento hierárquico foi utilizado o coeficiente aglomerativo (CA). Para estabelecer o número de agrupamento inicial foram empregados índices de validação. O método aglomerativo com melhor desempenho quanto ao (CA) foi de Ward com 0,94, indicando uma melhor força e qualidade entre as técnicas aglomerativos. Os índices de validação Davies Bouldin (DB), Dunn (D) e Silhouette (SIL), indicaram que a quantidade ideal de agrupamentos iniciais a ser formado são 2, todavia o índice PBM constatou que a formação ideal é com 4 grupos. Com relação aos municípios maior homogeneidade, verificou-se que na composição com 2 grupos, as observações mais similares foram m105(Salinópolis) e m109(Santa Izabel do Pará), seguido das observações m102 (Rio Maria) e m144 (Xinguara), todas inseridas no grupo 1.

The relative vertex clustering value--a new criterion for the fast discovery of functional modules in protein interaction networks.

BackgroundCellular processes are known to be modular and are realized by groups of proteins implicated in common biological functions. Such groups of proteins are called functional modules, and many community detection methods have been devised for their discovery from protein interaction networks (PINs) data. In current agglomerative clustering approaches, vertices with just a very few neighbors are often classified as separate clusters, which does not make sense biologically. Also, a major limitation of agglomerative techniques is that their computational efficiency do not scale well to large PINs. Finally, PIN data obtained from large scale experiments generally contain many false positives, and this makes it hard for agglomerative clustering methods to find the correct clusters, since they are known to be sensitive to noisy data.ResultsWe propose a local similarity premetric, the relative vertex clustering value, as a new criterion allowing to decide when a node can be added to a given node's cluster and which addresses the above three issues. Based on this criterion, we introduce a novel and very fast agglomerative clustering technique, FAC-PIN, for discovering functional modules and protein complexes from a PIN data.ConclusionsOur proposed FAC-PIN algorithm is applied to nine PIN data from eight different species including the yeast PIN, and the identified functional modules are validated using Gene Ontology (GO) annotations from DAVID Bioinformatics Resources. Identified protein complexes are also validated using experimentally verified complexes. Computational results show that FAC-PIN can discover functional modules or protein complexes from PINs more accurately and more efficiently than HC-PIN and CNM, the current state-of-the-art approaches for clustering PINs in an agglomerative manner.

Pruning boxes in a box-based classification method

In this work we address an extension of box clustering in supervised classification problems that makes use of optimization problems to refine the results obtained by agglomerative techniques. The central concept of box clustering is that of homogeneous boxes that give rise to overtrained classifiers under some conditions. Thus, we focus our attentions on the issue of pruning out redundant boxes, using the information gleaned from the other boxes generated under the hypothesis that such a choice would identify simpler models with good predictive power. We propose a pruning method based on an integer optimization problem and a family of sub problems derived from the main one. The overall performances are then compared to the accuracy levels of competing methods on a wide range of real data sets. The method has proven to be robust, making it possible to derive a more compact system of boxes in the instance space with good performance on training and test data.

Towards setting environmental water temperature guidelines: A South African example

Water temperature is a primary factor affecting the number and kinds of species in a stream. A key step towards including water temperatures in environmental flow assessments is to develop metrics which describe natural variability in a river's thermal regime. This is best achieved using time series analyses, where metrics are defined based either on time series disaggregation, or shapes of regimes defined using agglomerative techniques. The aim of this paper was to refine approaches in setting environmental water temperature guidelines for inclusion in defining environmental flows assessments. Annual water temperature series from 82 sites sampled across 48 rivers (mainstems and tributaries) in ten catchments in the southern Cape region of South Africa were described using 39 metrics based on the magnitude, frequency, duration and timing of thermal events. Sites were classified into thermal groups using their similarity in multivariate temperature regime and variation amongst groups along important temperature gradients examined. Deviation from a natural range of variability using a thermal confidence envelope is a suitable approach for broad evaluation of thermal guidelines. The approach presented can be applied at multiple levels of complexity to assess which elements of a thermal time series fall outside of reference conditions. Further steps in this approach are to link thermal patterns to biotic metrics, and gain a clearer understanding of interactions between flows, temperatures and biota, particularly below impoundments. Research on improving approaches in defining thermal regions is recommended.

Deterministic clustering of the available chemical space

Clustering of compound libraries using 2D binary fingerprints is a fundamental task in chemoinformatics and various methods have been described to solve it [1]. These methods can roughly be grouped into deterministic and non-deterministic approaches with two key-characteristics distinguishing them. First, the algorithmic complexity of deterministic approaches is more demanding whereas the non-deterministic methods often try to overcome this drawback by using heuristics to save time and memory. Second, deterministic clustering algorithms, especially agglomerative hierarchical techniques have been shown to yield good results and often perform better than non-deterministic approaches [2]. As a consequence, clustering of small to medium sized libraries with up to 1 million compounds is regularly performed using deterministic techniques whereas libraries comprising millions of compounds are mostly clustered using heuristics like k-means [3]. Here, we present a deterministic approach for clustering huge compound libraries based on all pairwise compound similarities. For this purpose, we use an extremely fast and flexible algorithm for similarity calculations, which we have developed to be purely CPU-based thus having no need for any specialized hardware. Using this similarity method, we implemented a workflow with the following steps. First, we create a set of unique input fingerprints by filtering duplicates that are then stored and finally remapped onto their representative clusters. Second, we calculate all pairwise similarities to construct a similarity network by applying a fixed Tanimoto threshold to select the edges to be inserted into the network. From this similarity network the connected subgraphs are extracted and forwarded to the last step. Finally, connected subgraphs exceeding a predefined size are hierarchically clustered. As a result, we show that our algorithm for similarity calculation is competitive to recently published CPU-based methods and can perform up to 380 million Tanimoto calculations per second on a current desktop computer. This efficient method allows our workflow to process medium to large libraries on current desktop computers within minutes. To finally demonstrate the power of our clustering workflow, we processed the commercially available chemical space comprising about 17 million compounds [4]. The entire clustering workflow took 63 hours on a compute server using 64 cores and 100 GB main memory to complete.

A model-independent approach to infer hierarchical codon substitution dynamics

BackgroundCodon substitution constitutes a fundamental process in molecular biology that has been studied extensively. However, prior studies rely on various assumptions, e.g. regarding the relevance of specific biochemical properties, or on conservation criteria for defining substitution groups. Ideally, one would instead like to analyze the substitution process in terms of raw dynamics, independently of underlying system specifics. In this paper we propose a method for doing this by identifying groups of codons and amino acids such that these groups imply closed dynamics. The approach relies on recently developed spectral and agglomerative techniques for identifying hierarchical organization in dynamical systems.ResultsWe have applied the techniques on an empirically derived Markov model of the codon substitution process that is provided in the literature. Without system specific knowledge of the substitution process, the techniques manage to "blindly" identify multiple levels of dynamics; from amino acid substitutions (via the standard genetic code) to higher order dynamics on the level of amino acid groups. We hypothesize that the acquired groups reflect earlier versions of the genetic code.ConclusionsThe results demonstrate the applicability of the techniques. Due to their generality, we believe that they can be used to coarse grain and identify hierarchical organization in a broad range of other biological systems and processes, such as protein interaction networks, genetic regulatory networks and food webs.

Local method for detecting communities

We propose a method of community detection that is computationally inexpensive and possesses physical significance to a member of a social network. This method is unlike many divisive and agglomerative techniques and is local in the sense that a community can be detected within a network without requiring knowledge of the entire network. A global application of this method is also introduced. Several artificial and real-world networks, including the famous Zachary karate club, are analyzed.

Generic climate maps: A strategic application of climate survey data?

Data have been gathered on 93 organizations and over 3030 individuals using a climate survey. This paper examines the related concepts of climate and culture and points to their importance as root metaphors for organizational analysis. Key research issues surrounding the use of climate surveys are highlighted and a range of strategic applications of climate survey data outlined. A twofold contribution to the literature is made by considering empirical and statistical approaches used in the field and by categorizing the various climates within and across industries. The Business and Organization Climate Index (BOCI) data are analysed statistically in order to examine the practicality and validity of using organizational climate maps as part of a typical management of change intervention. The findings show that the BOCI instrument can be used to create a reliable average profile with a high degree of inter-rater reliability. The methodological issues surrounding such assessments are discussed. Moreover, these organizational profiles can be classified, by use of agglomerative statistical techniques such as cluster analysis, into eight overarching climate maps. These generic climate maps reflect three negative, one neutral and four positive variations of climate. The membership of any particular organization in the cluster is not solely determined by industry sector. However, broad patterns of strategy and organizational process are reflected in the eight clusters. Given that climate surveys reflect individual perceptions, the eight climate maps may be as strong a measure of managers' and workers' cognitive frameworks and judgements as they are representations of organizational behaviour and action.

Twenty years of numerical syntaxonomy

The development of numerical syntaxonomy during its first 20 yr is reviewed. The use of methods of numerical classification and ordination is the dominating feature of the development. National and local phytosociological data banks were established, large data sets handled and many important vegetation monographs were methodically based on multivariate data analysis. Particularly the development in Italy, the Netherlands, Czechoslovakia, and Sweden contributed to new theoretical elements of numerical syntaxonomy. Ordination became a common tool of searching for reticulate synsystematic relations between community types. The most popular ordination techniques have been Principal Components Analysis and Detrended Correspondence Analysis. Hierarchical agglomerative techniques of clustering still prevail in classification, although the divisive strategy of TWINSPAN has also become an effective tool for phytosociological clustering and table sorting. Extensive program packages, also for personal computers have now become standard equipment for many vegetation scientists.

Monte Carlo comparison of six hierarchical clustering methods on random data

There is mounting evidence to suggest that the complete linkage method does the best clustering job among all hierarchical agglomerative techniques, particularly with respect to misclassification in samples from known multivariate normal distributions. However, clustering methods are notorious for discovering clusters on random data sets also. We compare six agglomerative hierarchical methods on univariate random data from uniform and standard normal distributions and find that the complete linkage method generally is best in not discovering false clusters. The criterion is the ratio of number of within-cluster distances to number of all distances at most equal to the maximum within-cluster distance.

A classification-ordination analysis of a Belgian mixed forest in the transition zone of two phytogeographical districts. II. Spring data

From 115 quadrats, laid out in the same way and on the same coordinates as described in a first paper, cover-abundance and height data were collected in april 1976 on 58 taxa, belonging to the upper and lower herb layer, on the moss layer as a whole and on the litter. They were submitted to four classification methods, namely Unweighted pair-group centroid technique (UPGC), normal Association analysis (NAA) , agglomerative normal and inverse Information analysis (NIA and IIA), Minimum variance clustering (MVC), and to four ordination methods, namely Beals' Polar ordination (POS), Simple ordination (SO), Optimized polar ordination (OPO) and Position vectors ordination (PVO). The only divisive method (NAA) produces more than sufficient results: they are slightly better with the 1%-stopping rule especially when no species reduction is involved. NAA and NIA give quite similar results. From the three agglomerative techniques, the greatest number of vegetation- clusters (7), recognizable in the field, has been obtained with MVC, the poorest picture however by UPGC. With regard to the forest-structure, the better results are with NIA, followed by MVC. Moreover. NIA applied in a quantitative way, is less appropriate. The outcome of species classification is not interpretable. The ordination results obtained by OPO and PVO are better as compared to those from POB and SO, as well in discovering gradients as clusters, the gradients particularly reflect changes in cover-abundance and height. In the ordination of vegetation quadrats, the total of the efficiency ratios extracted for the first three axes in PVO and GPO are respectively four and three times higher than with SO. Concerning the structure quadrats, the percentage extraction values are very high and very alike. Comparing the spring with the summer forest, UPGC distinguishes more identifiable vegetation types in the summer data, NAA on the contrary in the spring data.