Analysis Of Cytometry Data Research Articles

Alternatives to current flow cytometry data analysis for clinical and research studies.

Flow cytometry has well-established methods for data analysis based on traditional data collection techniques. These techniques typically involved manual insertion of tube samples into an instrument that, historically, could only measure 1-3 colors. The field has since evolved to incorporate new technologies for faster and highly automated sample preparation and data collection. For example, the use of microwell plates on benchtop instruments is now a standard on virtually every new instrument, and so users can easily accumulate multiple data sets quickly. Further, because the user must carefully define the layout of the plate, this information is already defined when considering the analytical process, expanding the opportunities for automated analysis. Advances in multi-parametric data collection, as demonstrated by the development of hyperspectral flow-cytometry, 20-40 color polychromatic flow cytometry, and mass cytometry (CyTOF), are game-changing. As data and assay complexity increase, so too does the complexity of data analysis. Complex data analysis is already a challenge to traditional flow cytometry software. New methods for reviewing large and complex data sets can provide rapid insight into processes difficult to define without more advanced analytical tools. In settings such as clinical labs where rapid and accurate data analysis is a priority, rapid, efficient and intuitive software is needed. This paper outlines opportunities for analysis of complex data sets using examples of multiplexed bead-based assays, drug screens and cell cycle analysis - any of which could become integrated into the clinical environment.

Cytofast: A workflow for visual and quantitative analysis of flow and mass cytometry data to discover immune signatures and correlations

Multi-parametric flow and mass cytometry allows exceptional high-resolution exploration of the cellular composition of the immune system. A large panel of computational tools have been developed to analyze the high-dimensional landscape of the data generated. Analysis frameworks such as FlowSOM or Cytosplore incorporate clustering and dimensionality reduction techniques and include algorithms allowing visualization of multi-parametric cytometric analysis. To additionally provide means to quantify specific cell clusters and correlations between samples, we developed an R-package, called cytofast, for further downstream analysis. Specifically, cytofast enables the visualization and quantification of cell clusters for an efficient discovery of cell populations associated with diseases or physiology. We used cytofast on mass and flow cytometry datasets based on the modulation of the immune system upon immunotherapy. With cytofast, we rapidly generated visual representations of group-related immune cell clusters and showed correlations with the immune system composition. We discovered macrophage subsets that significantly decrease upon cancer immunotherapy and distinct prime-boost effects of prophylactic vaccines on the myeloid compartment. Cytofast is a time-efficient tool for comprehensive cytometric analysis to reveal immune signatures and correlations. Cytofast is available at Bioconductor.

High throughput automated analysis of big flow cytometry data

The rapid expansion of flow cytometry applications has outpaced the functionality of traditional manual analysis tools used to interpret flow cytometry data. Scientists are faced with the daunting prospect of manually identifying interesting cell populations in 50-dimensional datasets, equalling the complexity previously only reached in mass cytometry. Data can no longer be analyzed or interpreted fully by manual approaches. While automated gating has been the focus of intense efforts, there are many significant additional steps to the analytical pipeline (e.g., cleaning the raw files, event outlier detection, extracting immunophenotypes). We review the components of a customized automated analysis pipeline that can be generally applied to large scale flow cytometry data. We demonstrate these methodologies on data collected by the International Mouse Phenotyping Consortium (IMPC).

Combination of mAb-AR20.5, anti-PD-L1 and PolyICLC inhibits tumor progression and prolongs survival of MUC1.Tg mice challenged with pancreatic tumors.

A substantial body of evidence suggests the existence of MUC1-specific antibodies and cytotoxic T cell activities in pancreatic cancer patients. However, tumor-induced immunosuppression renders these responses ineffective. The current study explores a novel therapeutic combination wherein tumor-bearing hosts can be immunologically primed with their own antigen, through opsonization with a tumor antigen-targeted antibody, mAb-AR20.5. We evaluated the efficacy of immunization with this antibody in combination with PolyICLC and anti-PD-L1. The therapeutic combination of mAb-AR20.5 + anti-PD-L1 + PolyICLC induced rejection of human MUC1 expressing tumors and provided a long-lasting, MUC1-specific cellular immune response, which could be adoptively transferred and shown to provide protection against tumor challenge in human MUC1 transgenic (MUC.Tg) mice. Furthermore, antibody depletion studies revealed that CD8 cells were effectors for the MUC1-specific immune response generated by the mAb-AR20.5 + anti-PD-L1 + PolyICLC combination. Multichromatic flow cytometry data analysis demonstrated a significant increase over time in circulating, activated CD8 T cells, CD3+CD4-CD8-(DN) T cells, and mature dendritic cells in mAb-AR20.5 + anti-PD-L1 + PolyICLC combination-treated, tumor-bearing mice, as compared to saline-treated control counterparts. Our study provides a proof of principle that an effective and long-lasting anti-tumor cellular immunity can be achieved in pancreatic tumor-bearing hosts against their own antigen (MUC1), which can be further potentiated using a vaccine adjuvant and an immune checkpoint inhibitor.

Visual analysis of mass cytometry data by hierarchical stochastic neighbour embedding reveals rare cell types

Mass cytometry allows high-resolution dissection of the cellular composition of the immune system. However, the high-dimensionality, large size, and non-linear structure of the data poses considerable challenges for the data analysis. In particular, dimensionality reduction-based techniques like t-SNE offer single-cell resolution but are limited in the number of cells that can be analyzed. Here we introduce Hierarchical Stochastic Neighbor Embedding (HSNE) for the analysis of mass cytometry data sets. HSNE constructs a hierarchy of non-linear similarities that can be interactively explored with a stepwise increase in detail up to the single-cell level. We apply HSNE to a study on gastrointestinal disorders and three other available mass cytometry data sets. We find that HSNE efficiently replicates previous observations and identifies rare cell populations that were previously missed due to downsampling. Thus, HSNE removes the scalability limit of conventional t-SNE analysis, a feature that makes it highly suitable for the analysis of massive high-dimensional data sets.

Biofabricating atherosclerotic plaques: In vitro engineering of a three-dimensional human fibroatheroma model

Atherosclerotic plaques are cholesterol-induced inflammatory niches accumulating in the vascular sub-endothelial space. Cellular and extracellular composition of human plaques is maneuvered by local inflammation that leads to alterations in the original vascular microenvironment and to the recruitment of an invading fibrous layer (fibroatharoma). In the present study we introduce a bioengineered three-dimensional model of human fibroatheroma (ps-plaque) assembled with a tailored hanging-drop protocol. Using vi-SNE based multidimensional flow cytometry data analysis we compared the myeloid cell-populations in ps-plaques to those in plaques isolated from human carotid arteries. We observed that plasmacytoid and activated dendritic cells are the main myeloid components of human carotid plaques and that both cell types are present in the biofabricated model. We found that low-density lipoproteins affect cell viability and contribute to population polarization in ps-plaques. The current work describes the first human bioengineered in vitro model of late atherosclerotic lesion for the investigation of atherosclerosis aetiopathogenesis.

Mass Cytometry and Topological Data Analysis Reveal Immune Parameters Associated with Complications after Allogeneic Stem Cell Transplantation

Human immune systems are variable, and immune responses are often unpredictable. Systems-level analyses offer increased power to sort patients on the basis of coordinated changes across immune cells and proteins. Allogeneic stem cell transplantation is a well-established form of immunotherapy whereby a donor immune system induces a graft-versus-leukemia response. This fails when the donor immune system regenerates improperly, leaving the patient susceptible to infections and leukemia relapse. We present a systems-level analysis by mass cytometry and serum profiling in 26 patients sampled 1, 2, 3, 6, and 12months after transplantation. Using a combination of machine learning and topological data analyses, we show that global immune signatures associated with clinical outcome can be revealed, even when patients are few and heterogeneous. This high-resolution systems immune monitoring approach holds the potential for improving the development and evaluation of immunotherapies in the future.

Automated Analysis of Flow Cytometry Data to Reduce Inter-Lab Variation in the Detection of Major Histocompatibility Complex Multimer-Binding T Cells.

Manual analysis of flow cytometry data and subjective gate-border decisions taken by individuals continue to be a source of variation in the assessment of antigen-specific T cells when comparing data across laboratories, and also over time in individual labs. Therefore, strategies to provide automated analysis of major histocompatibility complex (MHC) multimer-binding T cells represent an attractive solution to decrease subjectivity and technical variation. The challenge of using an automated analysis approach is that MHC multimer-binding T cell populations are often rare and therefore difficult to detect. We used a highly heterogeneous dataset from a recent MHC multimer proficiency panel to assess if MHC multimer-binding CD8+ T cells could be analyzed with computational solutions currently available, and if such analyses would reduce the technical variation across different laboratories. We used three different methods, FLOw Clustering without K (FLOCK), Scalable Weighted Iterative Flow-clustering Technique (SWIFT), and ReFlow to analyze flow cytometry data files from 28 laboratories. Each laboratory screened for antigen-responsive T cell populations with frequency ranging from 0.01 to 1.5% of lymphocytes within samples from two donors. Experience from this analysis shows that all three programs can be used for the identification of high to intermediate frequency of MHC multimer-binding T cell populations, with results very similar to that of manual gating. For the less frequent populations (<0.1% of live, single lymphocytes), SWIFT outperformed the other tools. As used in this study, none of the algorithms offered a completely automated pipeline for identification of MHC multimer populations, as varying degrees of human interventions were needed to complete the analysis. In this study, we demonstrate the feasibility of using automated analysis pipelines for assessing and identifying even rare populations of antigen-responsive T cells and discuss the main properties, differences, and advantages of the different methods tested.

Using Visualization of t-Distributed Stochastic Neighbor Embedding To Identify Immune Cell Subsets in Mouse Tumors.

High-dimensional flow cytometry is proving to be valuable for the study of subtle changes in tumor-associated immune cells. As flow panels become more complex, detection of minor immune cell populations by traditional gating using biaxial plots, or identification of populations that display small changes in multiple markers, may be overlooked. Visualization of t-distributed stochastic neighbor embedding (viSNE) is an unsupervised analytical tool designed to aid the analysis of high-dimensional cytometry data. In this study we use viSNE to analyze the simultaneous binding of 15 fluorophore-conjugated Abs and one cell viability probe to immune cells isolated from syngeneic mouse MB49 bladder tumors, spleens, and tumor-draining lymph nodes to identify patterns of anti-tumor immune responses. viSNE maps identified populations in multidimensional space of known immune cells, including T cells, B cells, eosinophils, neutrophils, dendritic cells, and NK cells. Based on the expression of CD86 and programmed cell death protein 1, CD8+ T cells were divided into distinct populations. Additionally, both CD8+ T cells and CD8+ dendritic cells were identified in the tumor microenvironment. Apparent differences between splenic and tumor polymorphonuclear cells/granulocytic myeloid-derived suppressor cells are due to the loss of CD44 upon enzymatic digestion of tumors. In conclusion, viSNE is a valuable tool for high-dimensional analysis of immune cells in tumor-bearing mice, which eliminates gating biases and identifies immune cell subsets that may be missed by traditional gating.

SCENERY: a web application for (causal) network reconstruction from cytometry data.

Flow and mass cytometry technologies can probe proteins as biological markers in thousands of individual cells simultaneously, providing unprecedented opportunities for reconstructing networks of protein interactions through machine learning algorithms. The network reconstruction (NR) problem has been well-studied by the machine learning community. However, the potentials of available methods remain largely unknown to the cytometry community, mainly due to their intrinsic complexity and the lack of comprehensive, powerful and easy-to-use NR software implementations specific for cytometry data. To bridge this gap, we present Single CEll NEtwork Reconstruction sYstem (SCENERY), a web server featuring several standard and advanced cytometry data analysis methods coupled with NR algorithms in a user-friendly, on-line environment. In SCENERY, users may upload their data and set their own study design. The server offers several data analysis options categorized into three classes of methods: data (pre)processing, statistical analysis and NR. The server also provides interactive visualization and download of results as ready-to-publish images or multimedia reports. Its core is modular and based on the widely-used and robust R platform allowing power users to extend its functionalities by submitting their own NR methods. SCENERY is available at scenery.csd.uoc.gr or http://mensxmachina.org/en/software/.

High output flow cytometry array profiling facilitates discriminant phenotyping of peripheral blood cells from patients with different forms of autoimmune uveitis

Abstract The information content of multi-parametric flow cytometry-based immune-phenotyping experiments is often underexploited given the paucity of adequate tools and strategies for large-scale unbiased data analysis. Here, we developed and applied a data analysis approach taking into account all mathematically possible combinations of markers in a given flow cytometry panel. We analyzed flow cytometry data generated from peripheral blood samples of healthy humans and patients with autoimmune uveitis in the setting of 2 different autoimmune diseases: Behcet’s disease and sarcoidosis. Original FACS data files were mined from Dryad Digital Repository http://dx.doi.org/10.5061/dryad.v6ste with reference to http://dx.doi.org/10.1371/journal.pcbi.1003215, and gated in bivariate plots. We employed combinatory mathematics to generate a matrix quantifying the representation of all possible cell populations using a given set of markers within the respective starting population and coded the algorithm to create a Java-based platform enabling the computation of large numbers of markers. Resulting metadata were visualized in a heat map approach. Our method enabled clustering of healthy vs diseased subjects using only 4 common markers (CD3, C8, CD197, and CD45), and of sarcoid vs Behcet’s patients with minimal error. Our approach demonstrates that multi-dimensional analysis of flow cytometry data allows meaningful large-scale screening of biologically relevant markers enabling classification and characterization of states of health and autoimmune disease. The approach is unbiased and has the potential to facilitate the discovery of cell populations with relevance as potential biomarkers or biological research targets.

Research Techniques Made Simple: Mass Cytometry Analysis Tools for Decrypting the Complexity of Biological Systems

Mass cytometry by time-of-flight experiments allow analysis of over 40 functional and phenotypic cellular markers simultaneously at the single-cell level. The data dimensionality escalation accentuates limitations, inherent to manual analysis, as being subjective, labor-intensive, slow, and often incapable of showing the detailed features of each unique cell within populations. The subsequent challenge of examining, visualizing, and presenting mass cytometry data has motivated continuous development of dimensionality reduction methods. As a result, an increasing recognition of the inherent diversity and complexity of cellular networks is emerging, with the discovery of unexpected cell subpopulations, hierarchies, and developmental pathways, such as those existing within the immune system. Here, we briefly review some frequently used and accessible mass cytometry data analysis tools, including principal component analysis (PCA); spanning-tree progression analysis of density-normalized events (SPADE); t-distributed stochastic neighbor embedding (t-SNE)-based visualization (viSNE); automatic classification of cellular expression by nonlinear stochastic embedding (ACCENSE); and cluster identification, characterization, and regression (CITRUS). Mass cytometry, used together with these innovative analytic tools, has the power to lead to key discoveries in investigative dermatology, including but not limited to identifying signaling phenotypes with predictive value for early diagnosis, prognosis, or relapse and a thorough characterization of intratumor heterogeneity and disease-resistant cell populations, that may ultimately unveil novel therapeutic approaches.

Static Magnetic Field Effect on Cell Alignment, Growth, and Differentiation in Human Cord-Derived Mesenchymal Stem Cells.

This investigation is performed to evaluate the impact of static magnetic field on the Cell growth alignment, and differentiation potential in Human Mesenchymal Stem cells derived from human newborn cords. In vitro-cultured mesenchymal stem cells derived from human newborn cords were exposed to SMF up to 24mT and compared with the control (unexposed) cultures. Viability was assessed via Trypan Blue staining and MTT assay. Cell cycle progression was studied after flow cytometry data analysis. Sox-2, Nanong, and Oct-4 Primers used for RT-PCR experiment. Morphological studies showed that the exposed cells were significantly aligned in parallel bundles in a correlation with the magnetic field lines. Viability measurements showed a significant reduction in cell viability which was noted after exposure to static magnetic field and initiated 36h after the end of exposure time. Flow cytometric data analysis confirmed a decrease in G1 phase cell population within the treated and cultured groups compared with the corresponding control samples. However, the induced changes were recovered in the cell cultures after the post-exposure culture recovery time which may be attributed to the cellular repair mechanisms. Furthermore, the proliferation rate and Oct-4 gene expression were reduced due to the 18mT static magnetic field exposure. The significant proliferation rate decrease accompanied by the Sox-2, Nanong, and Oct-4 gene expression decline, suggested the differentiation inducing effects of SMF exposure. Exposure to Static Magnetic fields up to 24mT affects mesenchymal stem cell alignment and proliferation rate as well as mRNA expression of Sox-2, Nanong, and Oct-4 genes, therefore can be considered as a new differentiation inducer in addition to the other stimulators.

Clinical Significance of Isolated Myeloperoxidase Expression in Pediatric B-Lymphoblastic Leukemia.

Diagnosis of B-cell acute lymphoblastic leukemia (B-ALL) requires immunophenotypic evidence of B-lineage and absence of specific myeloid or T-lineage markers. Rare cases of otherwise typical B-ALL express myeloperoxidase (MPO) detectable by flow cytometry with an absence of other myeloid markers, but the clinical significance of this finding is not well studied. A retrospective cohort analysis of flow cytometry and clinical data was performed to investigate the clinical outcome of this specific group of patients. Twenty-nine cases of otherwise typical B-ALL that expressed MPO by flow cytometry (B-ALL-isoMPO) without expression of other myeloid markers were identified. The B-ALL-isoMPO group had a significantly increased incidence of relapse (univariate log rank P = .0083; multivariate hazard ratio, 2.50; 95% confidence interval, 1.07-5.85; P = .034) and significantly worse event-free survival by univariate analysis (log rank P = .0066) compared with a reference group of patients with B-ALL from the same time period (n = 264). To our knowledge, this is the first report to document the clinical outcomes in a group of pediatric patients with B-ALL that expresses MPO in the absence of other myeloid markers. This group had an increased rate of relapse and a worse event-free survival than the patients with B-ALL who did not express MPO.

Toward deterministic and semiautomated SPADE analysis.

SPADE stands for spanning-tree progression analysis for density-normalized events. It combines downsampling, clustering and a minimum-spanning tree to provide an intuitive visualization of high-dimensional single-cell data, which assists with the interpretation of the cellular heterogeneity underlying the data. SPADE has been widely used for analysis of high-content flow cytometry data and CyTOF data. The downsampling and clustering components of SPADE are both stochastic, which lead to stochasticity in the tree visualization it generates. Running SPADE twice on the same data may generate two different tree structures. Although they typically lead to the same biological interpretation of subpopulations present in the data, robustness of the algorithm can be improved. Another avenue of improvement is the interpretation of the SPADE tree, which involves visual inspection of multiple colored versions of the tree based on expression of measured markers. This is essentially manual gating on the SPADE tree and can benefit from automated algorithms. This article presents improvements of SPADE in both aspects above, leading to a deterministic SPADE algorithm and a software implementation for semiautomated interpretation. © 2017 International Society for Advancement of Cytometry.

Flow Cytometric Single-Cell Identification of Populations in Synthetic Bacterial Communities.

Bacterial cells can be characterized in terms of their cell properties using flow cytometry. Flow cytometry is able to deliver multiparametric measurements of up to 50,000 cells per second. However, there has not yet been a thorough survey concerning the identification of the population to which bacterial single cells belong based on flow cytometry data. This paper not only aims to assess the quality of flow cytometry data when measuring bacterial populations, but also suggests an alternative approach for analyzing synthetic microbial communities. We created so-called in silico communities, which allow us to explore the possibilities of bacterial flow cytometry data using supervised machine learning techniques. We can identify single cells with an accuracy >90% for more than half of the communities consisting out of two bacterial populations. In order to assess to what extent an in silico community is representative for its synthetic counterpart, we created so-called abundance gradients, a combination of synthetic (i.e., in vitro) communities containing two bacterial populations in varying abundances. By showing that we are able to retrieve an abundance gradient using a combination of in silico communities and supervised machine learning techniques, we argue that in silico communities form a viable representation for synthetic bacterial communities, opening up new opportunities for the analysis of synthetic communities and bacterial flow cytometry data in general.

CymeR: cytometry analysis using KNIME, docker and R.

SummaryHere we present open-source software for the analysis of high-dimensional cytometry data using state of the art algorithms. Importantly, use of the software requires no programming ability, and output files can either be interrogated directly in CymeR or they can be used downstream with any other cytometric data analysis platform. Also, because we use Docker to integrate the multitude of components that form the basis of CymeR, we have additionally developed a proof-of-concept of how future open-source bioinformatic programs with graphical user interfaces could be developed.Availability and ImplementationCymeR is open-source software that ties several components into a single program that is perhaps best thought of as a self-contained data analysis operating system. Please see https://github.com/bmuchmore/CymeR/wiki for detailed installation instructions.

T Cell Repertoire after Alpha/Beta-T Cell Depleted Allogeneic Hematopoietic Stem Cell Transplantation in Pediatric Patients

Allogeneic transplantation of hematopoietic cells (HSCT) is an established method to treat different hematologic malignancies and disorders of hematopoietic and lymphoid system. Graft-versus-host-disease is one of the main risk factor for success of the procedure. Simultaneous depletion of alpha-beta T-cells and CD19+ cells in graft is the promising way to reduce the risk. The approach was recently introduced in clinical practice and many aspects of immune system reconstitution are still unknown. We applied improved technology for T cell receptor (TCR) repertoire sequencing to study origin and dynamics of T cell clones during 1 year follow-up period after allogeneic TCRαβ/CD19-depleted HSCT in children.We performed TCR repertoire sequencing for peripheral blood samples of patients before HSCT, at 2, 6 and 12 months after HSCT (n=21, 21, 17, 16 respectively), and for respective donor blood apheresis samples before abT/CD19 depletion. Twelve of the patients were diagnosed with acute leukemia and the others with non-malignant inherited and acquired blood disorders. For each patient data on recipient’s T cell chimerism and counts of CD3+, naïve CD3+, alpha-beta T-cells and recent thymic emigrants (RTE) have been collected during 1 year follow-up period.Barcoding of each original TCR mRNA molecule passed to massive parallel sequencing allowed us to: (1) reduce sample preparation biases and quantitatively reconstruct of TCR repertoires; (2) equalize repertoire data analysis depth which is absolutely necessary for correct comparison of samples; (3) prevent risk of cross-contamination between samples and increase confidence of T clone origin determination.Two months after TCRαβ/CD19-depleted HSCT T cell repertoire mostly consists of several hundreds highly abundant clones. For patients with low recipient T cell chimerism from 13 to 504 largest T-cell clones (median 255, IQR 219, n=9, T cell chimerism <=20%) represented 80% of all T cells in peripheral blood. For comparison in healthy age-matched donors we found from 32,000 to 47,000 largest T-cell clones in identical analysis (median 43191, IQR 6493, n=14, data from Britanova O.V. et.al., JI 2014). The overall diversity at d60 after HSCT was also much less compared with the healthy subjects. We also found that most expanded T cell clones at d60 do not represent just a replica of the most expanded clones in graft samples, originating from low-abundant graft T cell clones.The diversity of T repertoire early after HSCT positively correlated with recipient T cell chimerism (the diversity was higher for those patients with higher percentage of recipient’s T cells). Also patients with low chimerism had higher number of T clones originating from the graft than from d0 pre-transplant recipient repertoire in contrast to the patients with high T cell chimerism who had inverse ratio (median number of patient’s clonotypes shared between graft and d0 was 56 or 3 for patients with low or absent chimerism (IQR = 24 or 19.25, n = 5) and 21.5 or 321.5 for patients with T cell chimerism >2% (IQR = 46.5 or 724.75, n = 10)). In addition CD4+ RTE count was higher for patients with high T cell chimerism. This observation was additionally confirmed by analysis of flow cytometry data for the expanded cohort of 105 patients at d60 after αβT-cell depleted HSCT (Wilcoxon rank sum test p-value = 0.002).Our results demonstrate that early after αβT-cell depleted HSCT repertoire of T cells are extremely skewed and unlikely able protect recipient efficiently. Observed recovery of T cell count mostly results from expansion of a few clones that have to divide intensely for the whole 60 days period in order to achieve the observed counts. Early reconstitution of TCR diversity and RTE counts in patients with substantial recipient T cell chimerism is mostly explained by surviving recipient T cells and intrathymic T cell progenitors, respectively.This work was supported by the Russian Science Foundation project №14-35-00105. Zvyagin I. is supported by grant MK-4583.2015.4. DisclosuresNo relevant conflicts of interest to declare.

Volume 89A, Number 11, November 2016 Cover Image

AbstractThe Support Vector Machines are coming! The Support Vector Machines are coming! No worries, they are here to replace a complex and sometimes irreproducible manual or semi‐automated flow cytometry data analysis of bone marrow samples. Read not one, not two, but three accompanying articles by Voigt and coworkers in this issue of Cytometry Part A.Cover design by Bärbel Beran [www.beran-design.de].

Cluster stability in the analysis of mass cytometry data.

Manual gating has been traditionally applied to cytometry data sets to identify cells based on protein expression. The advent of mass cytometry allows for a higher number of proteins to be simultaneously measured on cells, therefore providing a means to define cell clusters in a high dimensional expression space. This enhancement, whilst opening unprecedented opportunities for single cell-level analyses, makes the incremental replacement of manual gating with automated clustering a compelling need. To this aim many methods have been implemented and their successful applications demonstrated in different settings. However, the reproducibility of automatically generated clusters is proving challenging and an analytical framework to distinguish spurious clusters from more stable entities, and presumably more biologically relevant ones, is still missing. One way to estimate cell clusters' stability is the evaluation of their consistent re-occurrence within- and between-algorithms, a metric that is commonly used to evaluate results from gene expression. Herein we report the usage and importance of cluster stability evaluations, when applied to results generated from three popular clustering algorithms - SPADE, FLOCK and PhenoGraph - run on four different data sets. These algorithms were shown to generate clusters with various degrees of statistical stability, many of them being unstable. By comparing the results of automated clustering with manually gated populations, we illustrate how information on cluster stability can assist towards a more rigorous and informed interpretation of clustering results. We also explore the relationships between statistical stability and other properties such as clusters' compactness and isolation, demonstrating that whilst cluster stability is linked to other properties it cannot be reliably predicted by any of them. Our study proposes the introduction of cluster stability as a necessary checkpoint for cluster interpretation and contributes to the construction of a more systematic and standardized analytical framework for the assessment of cytometry clustering results. © 2016 International Society for Advancement of Cytometry.