Use Of Hierarchical Clustering Research Articles

Diet and trophic structure of fishes in the Barents Sea: Effects of size within (ontogenetic) and between species

A large data set on stomach content (over 27,000 individual fish) – collected in the Barents Sea in 2015 – was used to examine ontogenetic and interspecific changes in diet with size for 35 fish species. The analysis combined the use of hierarchical clustering and random forest. Two different diet metrics were used (% weight of prey types per weight of stomach content or per weight of fish, the latter reflecting also feeding intensity), and data were analysed based on average diet of predator groups (species and size groups) across the entire sample series (from the whole Barents Sea in different seasons), or for data broken down by geographical areas and seasons. Similar trophic groups (clusters) were identified for the various data sets suggesting that the results on trophic structure were robust. The trophic groups including size information were broadly similar to groups found earlier with data averaged at species level. Hierarchical clustering produces a hierarchy of trophic groups (or trophic guilds) at various levels of diet dissimilarity. With 12 clusters identified, one cluster had fish as dominant prey, and 5–6 others had either plankton or benthos as dominant prey. The clusters tended to be distinct and homogenous with one dominant prey category in average diet (often > 60 %). This was especially the case for the plankton clusters, with copepods, euphausiids, hyperiid amphipods, or gelatinous zooplankton as dominant prey in each of 4 trophic groups. The benthos clusters tended to be less dissimilar with more overlap in diet composition, with predominance of either hard-shelled forms (echinoderms and molluscs) or softer prey (polychaetes and crustaceans) for groups of clusters. There were clear patterns of ontogenetic shifts in fish diet. Fish that clustered as piscivores at larger size tended to grow from being planktivores when smaller. Smaller species of planktivores or benthivores shifted position among trophic groups but remained within the categories of planktivores or benthivores, respectively. Taxonomy (species) tended to be more important than size for explaining the diet composition of the different clusters of fish predators.

Surrogate-assisted sine Phasmatodea population evolution algorithm applied to 3D coverage of mobile nodes

AbstractDeploying static wireless sensor nodes is prone to network coverage gaps, resulting in poor network coverage. In this paper, an attempt is made to improve the network coverage by moving the locations of the nodes. A surrogate-assisted sine Phasmatodea population evolution algorithm (SASPPE) is used to evaluate the network coverage. A $$50 \times 50$$ 50 × 50 hill simulation environment was tested for the number of nodes of 30 and 40 and radii of 3, 5 and 7, respectively. The results show that the SASPPE algorithm has the highest coverage, which can be up to 23.624% higher than the PPE algorithm, and up to 5.196% higher than the PPE algorithm, ceteris paribus. The SASPPE algorithm mixes the GSAM with LSAMs, which balances the computational cost of the algorithm and the algorithm’s ability to find optimal results. The use of hierarchical clustering enhances the stable type of the LSAMs. In addition, LSAMs are easy to fall into local optimality when they are modeled with local data, and the use of sine Phasmatodea population evolution algorithm (Sine-PPE) for searching in LSAMs alleviates the time for the algorithm to fall into local optimality. On 30D, 50D, and 100D, the proposed algorithm was tested by 7 test functions. The results show that the algorithm has significant advantages on most functions.

Modal strain monitoring of the old Nieuwebrugstraat Bridge: Local damage versus temperature effects

Vibration-Based Monitoring (VBM) from modal strains is a promising alternative to traditional VBM methods, which are primarily based on exclusively monitoring natural frequencies. The advantages of modal strains are that they offer a lower sensitivity to temperature and a higher sensitivity to small-scale damage, when compared to natural frequencies, rendering them ideal for detecting damage at an early stage. Furthermore, modal strains can be obtained in a dense grid with a relatively low cost, when fiber-optic sensors, such as fiber-Bragg gratings (FBG) are employed, which enables the monitoring of several critical locations and increases the possibility of early damage detection and localization. Modal strains can be obtained with high accuracy and precision thanks to a recently developed methodology that makes use of FBG sensors and of a high resolution acquisition system. The temperature sensitivity of modal strains has been investigated in the field but their damage sensitivity has previously only been investigated through laboratory experiments and numerical simulations. In this study, a 110-years-old steel railway bridge is monitored with 80 FBGs. Its modal strains and natural frequencies are automatically identified with the use of hierarchical clustering from ambient and operational dynamic strains. Before the scheduled replacement of the bridge, artificial damage is introduced by means of local cuts, simulating fatigue cracks, in order to investigate the damage detection and localization capabilities of modal strains on a full-size civil structure for the first time. The influence of temperature and operational factors is also investigated and compared with that of damage.

Intelligent Deep Learning Enabled Human Activity Recognition for Improved Medical Services

Human Activity Recognition (HAR) has been made simple in recent years, thanks to recent advancements made in Artificial Intelligence (AI) techniques. These techniques are applied in several areas like security, surveillance, healthcare, human-robot interaction, and entertainment. Since wearable sensor-based HAR system includes in-built sensors, human activities can be categorized based on sensor values. Further, it can also be employed in other applications such as gait diagnosis, observation of children/adult’s cognitive nature, stroke-patient hospital direction, Epilepsy and Parkinson’s disease examination, etc. Recently-developed Artificial Intelligence (AI) techniques, especially Deep Learning (DL) models can be deployed to accomplish effective outcomes on HAR process. With this motivation, the current research paper focuses on designing Intelligent Hyperparameter Tuned Deep Learning-based HAR (IHPTDL-HAR) technique in healthcare environment. The proposed IHPTDL-HAR technique aims at recognizing the human actions in healthcare environment and helps the patients in managing their healthcare service. In addition, the presented model makes use of Hierarchical Clustering (HC)-based outlier detection technique to remove the outliers. IHPTDL-HAR technique incorporates DL-based Deep Belief Network (DBN) model to recognize the activities of users. Moreover, Harris Hawks Optimization (HHO) algorithm is used for hyperparameter tuning of DBN model. Finally, a comprehensive experimental analysis was conducted upon benchmark dataset and the results were examined under different aspects. The experimental results demonstrate that the proposed IHPTDL-HAR technique is a superior performer compared to other recent techniques under different measures.

Topic Analysis of Nursing Research Using Co-Word Analysis.

Background:Nursing is one of the most important areas of medical sciences whose developments including its scientific publications can influence health care. Therefore, the aim of this study was to investigate nursing articles published from 2013 to 2018, and to provide a comprehensive view of common topical clusters in this research area.Materials and Methods:In this practical research, bibliometrics method and co-word analysis technique are used. The study population included all the articles in nursing area indexed in Web of Science from 2013 to 2018. The bibliometrics software, including BibExcel, UCINET, and SPSS was used to analyze the data.Results:Results indicated that the most frequently used nursing words in nursing research articles were “Quality of life,” “Aged,” “Education,” and “Nursing.” Moreover, the pairs such as “Anxiety-Depression”, “Education, Nursing-Students, Nursing” and “Depression--Quality of life” were the most frequent co-occurrences. The use of hierarchical clustering led to the formation of seven topical clusters in Nursing: “Nursing care for the Aged,” “Self-care,” “Physical, emotional, and social support, “Mother and child health,” “Preventing nursing care,” “Nursing profession Research,” and “Quality of nursing care.”Conclusions:The growth of nursing scientific productions is an indicative of the importance of this subject area in healthcare services; however, there is no balanced growth in various subjects.

A New Ensemble Method for Detecting Anomalies in Gene Expression Matrices

One of the main problems in the analysis of real data is often related to the presence of anomalies. Namely, anomalous cases can both spoil the resulting analysis and contain valuable information at the same time. In both cases, the ability to detect these occurrences is very important. In the biomedical field, a correct identification of outliers could allow the development of new biological hypotheses that are not considered when looking at experimental biological data. In this work, we address the problem of detecting outliers in gene expression data, focusing on microarray analysis. We propose an ensemble approach for detecting anomalies in gene expression matrices based on the use of Hierarchical Clustering and Robust Principal Component Analysis, which allows us to derive a novel pseudo-mathematical classification of anomalies.

Two-stage topic modelling of scientific publications: A case study of University of Nairobi, Kenya.

Unsupervised statistical analysis of unstructured data has gained wide acceptance especially in natural language processing and text mining domains. Topic modelling with Latent Dirichlet Allocation is one such statistical tool that has been successfully applied to synthesize collections of legal, biomedical documents and journalistic topics. We applied a novel two-stage topic modelling approach and illustrated the methodology with data from a collection of published abstracts from the University of Nairobi, Kenya. In the first stage, topic modelling with Latent Dirichlet Allocation was applied to derive the per-document topic probabilities. To more succinctly present the topics, in the second stage, hierarchical clustering with Hellinger distance was applied to derive the final clusters of topics. The analysis showed that dominant research themes in the university include: HIV and malaria research, research on agricultural and veterinary services as well as cross-cutting themes in humanities and social sciences. Further, the use of hierarchical clustering in the second stage reduces the discovered latent topics to clusters of homogeneous topics.

Preliminary evaluation of a novel nine-biomarker profile for the prediction of autism spectrum disorder.

Autism spectrum disorder (ASD) is a complex group of heterogeneous neurodevelopmental disorders the prevalence of which has been in the rise in the past decade. In an attempt to better target the basic causes of ASD for diagnosis and treatment, efforts to identify reliable biomarkers related to the body's metabolism are increasing. Despite an increase in identifying biomarkers in ASD, there are none so far with enough evidence to be used in routine clinical examination, unless medical illness is suspected. Promising biomarkers include those of mitochondrial dysfunction, oxidative stress, energy metabolism, and apoptosis. Sodium (Na+), Potassium (K+), glutathione (GSH), glutathione-s-transferase (GST), Creatine kinase (CK), lactate dehydrogenase (LDH), Coenzyme Q10, and melatonin (MLTN) were evaluated in 13 participants with ASD and 24 age-matched healthy controls. Additionally, five ratios, which include Na+/K+, GSH:GST, CK:Cas7, CoQ10: Cas 7, and Cas7:MLTN, were tested to measure their predictive values in discriminating between autistic individuals and controls. These markers, either in absolute values, as five ratios, or combined (9 markers + 5 ratios) were subjected to a principal component analysis and multidimensional scaling (MDS), and hierarchical clustering, which are helpful statistical tools in the field of biomarkers. Our data demonstrated that both PCA and MDS analysis were effective in separating autistic from control subjects completely. This was also confirmed through the use of hierarchical clustering, which showed complete separation of the autistic and control groups based on nine biomarkers, five biomarker ratios, or a combined profile. Excellent predictive value of the measured profile was obtained using the receiver operating characteristics analysis, which showed an area under the curve of 1. The availability of an improved predictive profile, represented by nine biomarkers plus the five ratios, inter-related different etiological mechanisms in ASD and would be valuable in providing greater recognition of the altered biological pathways in ASD. Our predictive profile could be used for the diagnosis and intervention of ASD.

A jump model for credit default swaps with hierarchical clustering

The return time series of Credit Default Swaps (CDS) display possibly the highest excess kurtosis and skewness of any asset class in capital markets. Capturing this requires a departure from classical modeling techniques. Here, the premise is that CDS prices jump and those jumps cluster. Initially, a model is proposed that is driven by a diffusion and a Hawkes process to reproduce the clustering of shocks. The new element of the model is the use of hierarchical clustering to calibrate the distribution of the jumps. Partitioning the shocks identifies shifts in the volatility regime of the CDS through time, which are shown to consistently correspond to both systemic and idiosyncratic market events. A piecewise definition of a double exponential distribution for the jumps, corresponding to the identified clusters, is employed to replicate the regime switching. Subsequently, this demonstrably improves the accuracy of the calibration. The model is fitted to all CDS with more than 2000 closing prices from 2001 to 2014; a total of 1652 individual time series. For each credit, a peak-over-threshold approach is applied to the return series to separate the data into those returns that can broadly be classified as log-normal and those that can be characterized as jumps. By explicitly linking the results to market traded volumes, the results indicate a clear split of the CDS into liquid and illiquid names. A key finding is that for the illiquid CDS, it is difficult to justify the continuous component of the model. For the low liquidity names, the Hawkes process is shown to capture the price-action without a diffusion. A Markov chain is then introduced into the simulation to replicate the clustering and regime switching.

Efficient methods and readily customizable libraries for managing complexity of large networks.

BackgroundOne common problem in visualizing real-life networks, including biological pathways, is the large size of these networks. Often times, users find themselves facing slow, non-scaling operations due to network size, if not a “hairball” network, hindering effective analysis. One extremely useful method for reducing complexity of large networks is the use of hierarchical clustering and nesting, and applying expand-collapse operations on demand during analysis. Another such method is hiding currently unnecessary details, to later gradually reveal on demand. Major challenges when applying complexity reduction operations on large networks include efficiency and maintaining the user’s mental map of the drawing.ResultsWe developed specialized incremental layout methods for preserving a user’s mental map while managing complexity of large networks through expand-collapse and hide-show operations. We also developed open-source JavaScript libraries as plug-ins to the web based graph visualization library named Cytsocape.js to implement these methods as complexity management operations. Through efficient specialized algorithms provided by these extensions, one can collapse or hide desired parts of a network, yielding potentially much smaller networks, making them more suitable for interactive visual analysis.ConclusionThis work fills an important gap by making efficient implementations of some already known complexity management techniques freely available to tool developers through a couple of open source, customizable software libraries, and by introducing some heuristics which can be applied upon such complexity management techniques to ensure preserving mental map of users.

The use of hierarchical clustering for the design of optimized monitoring networks

Abstract. Associativity analysis is a powerful tool to deal with large-scale datasets by clustering the data on the basis of (dis)similarity and can be used to assess the efficacy and design of air quality monitoring networks. We describe here our use of Kolmogorov–Zurbenko filtering and hierarchical clustering of NO2 and SO2 passive and continuous monitoring data to analyse and optimize air quality networks for these species in the province of Alberta, Canada. The methodology applied in this study assesses dissimilarity between monitoring station time series based on two metrics: 1−R, R being the Pearson correlation coefficient, and the Euclidean distance; we find that both should be used in evaluating monitoring site similarity. We have combined the analytic power of hierarchical clustering with the spatial information provided by deterministic air quality model results, using the gridded time series of model output as potential station locations, as a proxy for assessing monitoring network design and for network optimization. We demonstrate that clustering results depend on the air contaminant analysed, reflecting the difference in the respective emission sources of SO2 and NO2 in the region under study. Our work shows that much of the signal identifying the sources of NO2 and SO2 emissions resides in shorter timescales (hourly to daily) due to short-term variation of concentrations and that longer-term averages in data collection may lose the information needed to identify local sources. However, the methodology identifies stations mainly influenced by seasonality, if larger timescales (weekly to monthly) are considered. We have performed the first dissimilarity analysis based on gridded air quality model output and have shown that the methodology is capable of generating maps of subregions within which a single station will represent the entire subregion, to a given level of dissimilarity. We have also shown that our approach is capable of identifying different sampling methodologies as well as outliers (stations' time series which are markedly different from all others in a given dataset).

753 Use of hierarchical clustering and principal component analysis for deep phenotyping of patients with mutations in XPD (ERCC2): trichothiodystrophy (TTD), xeroderma pigmentosum (XP) and XP/TTD

753 Use of hierarchical clustering and principal component analysis for deep phenotyping of patients with mutations in XPD (ERCC2): trichothiodystrophy (TTD), xeroderma pigmentosum (XP) and XP/TTD

A Jump Model for Credit Default Swaps with Hierarchical Clustering

The return time series of Credit Default Swaps (CDS) display possibly the highest excess kurtosis and skewness of any asset class in capital markets. Capturing this requires a departure from classical modelling techniques. Here, the premise is that CDS prices jump and those jumps cluster. Initially, a model is proposed that is driven by a diffusion and a Hawkes process to reproduce the clustering of shocks. The new element of the model is the use of hierarchical clustering to calibrate the distribution of the jumps. Partitioning the shocks identifies shifts in the volatility regime of the CDS through time, which are shown to consistently correspond to both systemic and idiosyncratic market events. A piecewise definition of a double exponential distribution for the jumps, corresponding to the identified clusters, is employed to replicate the regime switching. Subsequently, this demonstrably improves the accuracy of the calibration. The model is fitted to all CDS with more than 2000 closing prices from 2001 to 2014; a total of 1652 individual time series. For each credit, a peak-over-threshold approach is applied to the return series to separate the data into those returns that can broadly be classified as log-normal and those that can be characterized as jumps. By explicitly linking the results to market traded volumes, the results indicate a clear split of the CDS into liquid and illiquid names. A key finding is that for the illiquid CDS, it is difficult to justify the continuous component of the model. For the low liquidity names, the Hawkes process is shown to capture the price-action without a diffusion. A Markov chain is then introduced into the simulation to replicate the clustering and regime switching.

Multiplate® evaluation of acetylsalicylic acid efficacy in carotid surgery: routine and genetic influencing factors

Background Acetylsalicylic acid (ASA) is recommended before, during and after carotid endarterectomy (CEA). The efficacy of ASA is influenced by numerous biological and genotypic factors. Objectives To determine the biological efficacy of ASA by using the Multiplate® method, and to explore the biological parameters and genomic factors influencing this efficacy. Methods This descriptive cross-sectional study included all patients scheduled for CEA between January 2012 and April 2013. Multiplate® tests were performed at day 0 and day 30. A set of 66 single-nucleotide polymorphisms (SNPs) from 38 genes or DNA regions were selected and studied along with phenotypic parameters by the use of hierarchical clustering (HC) for multidimensional data management. Results Fifty-five patients receiving ASA were analyzed. Of the patients, 95% were found to be sensitive to ASA, with values under the threshold of normality (400 AU min-1 ). However, there were notable differences in residual aggregation among subjects over a wide range. HC revealed four subclusters comprising three categories of parameters: (i) routine and functional parameters - in ASA-treated patients, the ASPItest was highly linked to the ADPtest, to platelet count, and, to a lesser extent, to fibrinogen and hematocrit; (ii) polymorphisms in genes involved in ASA absorption and in the arachidonic acid pathway (ABCB1 and COX-1); and (iii) polymorphisms in genes modulating basal platelet function, i.e. TBXA2R, ADRA2A, PEAR1, ITGA2 and ITGB1. Conclusion Most patients treated with ASA before CEA were sensitive to it, according to Multiplate® ASPItest results. Genomic factors influencing this efficacy are SNPs involved in ASA absorption and metabolic pathway, and in modulations in basal platelet function.

Applied fault detection and diagnosis for industrial gas turbine systems

The paper presents readily implementable approaches for fault detection and diagnosis (FDD) based on measurements from multiple sensor groups, for industrial systems. Specifically, the use of hierarchical clustering (HC) and self-organizing map neural networks (SOMNNs) are shown to provide robust and user-friendly tools for application to industrial gas turbine (IGT) systems. HC fingerprints are found for normal operation, and FDD is achieved by monitoring cluster changes occurring in the resulting dendrograms. Similarly, fingerprints of operational behaviour are also obtained using SOMNN based classification maps (CMs) that are initially determined during normal operation, and FDD is performed by detecting changes in their CMs. The proposed methods are shown to be capable of FDD from a large group of sensors that measure a variety of physical quantities. A key feature of the paper is the development of techniques to accommodate transient system operation, which can often lead to false-alarms being triggered when using traditional techniques if the monitoring algorithms are not first desensitized. Case studies showing the efficacy of the techniques for detecting sensor faults, bearing tilt pad wear and early stage pre-chamber burnout, are included. The presented techniques are now being applied operationally and monitoring IGTs in various regions of the world.

Information support diagnostic-medical action in medicine

The Purpose of the work is a development of information technology for diagnostic and treatment activities steps complex estimation for increasing of their efficiency and minimization of the risk of doctor’s mistakes. The Formalized stages diagnostic-medical process at development computer decision support system in medicine. The risks of incorrect decision appear at the stage of diagnosis: an error of the first kind α and an error of the second kind β are defined with the location of the training set objects in the feature space without considering their impact on the stage of selecting the desired medical action. To minimize the risk of incorrect medical measures due to incorrect diagnosis found the relation between the error diagnostics and the consequences of this error in the appointment of medical action. Transition from the traditional space of marks to the medical action space is offered to minimize the risk. The components medical action space are binary variables (0 – absent, 1 – present), and every diagnosable condition is represented with a point in the i-th vertex of the hypercube. The use of hierarchical clustering with the criterion of minimum aggregate relations (the search for the minimum cut) in the medical action space for the synthesis of the decision tree provides minimum risk of decision-making in integrated assessment of diagnostic and medical action. In the work the correction method of thresholds of uncertainty interval in solving rule is proposed for an integrated risk assessment of medical errors that arise at both stages of diagnostic and treatment process, including at step medical action. Developed information technologies allow to minimize the risk of medical errors, to improve the reliability and validity of the decisions of the decision maker, and can be adapted to the different subject fields of medicine

The good, the bad and the implications of profiling mobile broadband coverage

Pervasive coverage and continuous connectivity of Mobile Broadband (MBB) networks are common goals for regulators and operators. Given the increasing heterogeneity of technologies in the last mile of MBB networks, further support for seamless connectivity across multiple network types relies on understanding the prevalent network coverage profiles that capture different available technologies in an area. Correlating these coverage profiles with network performance metrics is of great importance in order to forestall disturbances for applications running on top of MBB networks. In this paper, we aim to profile MBB coverage and its performance implications from the end-user’s perspective along critical transport infrastructures (i.e., railways in Norway). For this, we deploy custom measurement nodes on-board five Norwegian inter-city trains and we collect a unique geo-tagged dataset along the train routes. We then build a coverage mosaic, where we divide the routes into segments and analyze the coverage of individual operators in each segment. We propose and evaluate the use of hierarchical clustering to describe prevalent coverage profiles of MBB networks along the train routes and classify each segment accordingly. We further analyze the areas we classify with each profile and assess the packet-loss and HTTP download performance of the networks in those areas.

Coordinated Robot Navigation via Hierarchical Clustering

We introduce the use of hierarchical clustering for relaxed deterministic coordination and control of multiple robots. Traditionally, an unsupervised learning method, hierarchical clustering offers a formalism for identifying and representing spatially cohesive and segregated robot groups at different resolutions by relating the continuous space of configurations to the combinatorial space of trees. We formalize and exploit this relation, developing computationally effective reactive algorithms for navigating through the combinatorial space in concert with geometric realizations for a particular choice of the hierarchical clustering method. These constructions yield computationally effective vector field planners for both hierarchically invariant as well as transitional navigation in the configuration space. We apply these methods to the centralized coordination and control of $n$ perfectly sensed and actuated Euclidean spheres in a $d$ -dimensional ambient space (for arbitrary $n$ and $d$ ). Given a desired configuration supporting a desired hierarchy, we construct a hybrid controller that is quadratic in $n$ and algebraic in $d$ and prove that its execution brings all but a measure zero set of initial configurations to the desired goal, with the guarantee of no collisions along the way.

Design of rule-based models through information granulation

In this study, we explore the combination of two well defined topics in fuzzy systems research: fuzzy rule based systems, and information granulation. Rule based systems are a powerful and well-studied form of knowledge representation, due to their approximation abilities and interpretability. In recent years, these types of systems have become increasingly powerful with regards to modeling accuracy; however, many of these improvements come at the cost of model interpretability. This recent direction of research has left an unexplored avenue towards the generation of increasingly interpretable fuzzy rule based models, which we intend to explore. Information granulation is a relatively new, yet very promising area of research in human centric systems. As a form of knowledge representation, information granulation is very well suited to fuzzy rule based systems, where rules represent linguistic quantities in a, intuitively understandable format. It is notable that the combination of these two concepts has been left largely unstudied. We aim to explore this union by defining a methodology for the construction of a partially granular fuzzy rule based model. The aim of this novel model format is to provide a first step in the improvement of fuzzy model interpretability, through the use of information granulation. We are additionally interested in studying new ways of generating fuzzy rules; hence, we will also look at the use of hierarchical clustering as a potential alternative to the tried and tested Fuzzy C Means clustering algorithm. The models created using hierarchical clustering are then compared with those generated using Fuzzy C Means to evaluate the effectiveness of this algorithm. As a result of these experiments, we demonstrate that partially granular fuzzy rules are capable of providing a significant improvement to fuzzy rule interpretability, and we believe that granular fuzzy models present an exciting avenue of future research in human centric systems.

Improving the Decision Value of Hierarchical Text Clustering Using Term Overlap Detection

Humans are used to expressing themselves with written language and language provides a medium with which we can describe our experiences in detail incorporating individuality. Even though documents provide a rich source of information, it becomes very difficult to identify, extract, summarize and search when vast amounts of documents are collected especially over time. Document clustering is a technique that has been widely used to group documents based on similarity of content represented by the words used. Once key groups are identified further drill down into sub-groupings is facilitated by the use of hierarchical clustering. Clustering and hierarchical clustering are very useful when applied to numerical and categorical data and cluster accuracy and purity measures exist to evaluate the outcomes of a clustering exercise. Although the same measures have been applied to text clustering, text clusters are based on words or terms which can be repeated across documents associated with different topics. Therefore text data cannot be considered as a direct ‘coding’ of a particular experience or situation in contrast to numerical and categorical data and term overlap is a very common characteristic in text clustering. In this paper we propose a new technique and methodology for term overlap capture from text documents, highlighting the different situations such overlap could signify and discuss why such understanding is important for obtaining value from text clustering. Experiments were conducted using a widely used text document collection where the proposed methodology allowed exploring the term diversity for a given document collection and obtain clusters with minimum term overlap.