Internal Validation Measures Research Articles

Employing cluster analysis in defining groundwater wells patterns in Rafah

Clustering analysis techniques used in identifying homogeneity of groundwater wells patterns in terms of Chloride, Nitrate, and TDS. Data was collected through the laboratories of Palestinian Water Authority in the Rafah 2020. R-Programming is used for data analysis. Kolmogorov-Smirnov test is used to test data normality. Hierarchical clustering analysis applied to generate a cluster tree (r>0.75) in correlation matrix. Agglomerative Hierarchical Clustering is used to classify with “average” method and found that there are two clusters. First cluster with 28 wells and Second cluster with 9 wells. Test of homogeneity between the two clusters groups using T-test for the hierarchical clustering and found that there are significant differences in means for TDS and for Chloride between first cluster and second cluster with a significant level less than 0.01. While for Nitrates, also, there are significant differences in means between the first cluster and second cluster with a significant level less than 0.05. Different stability validation measures carried out for data concludes that for the APN and ADM measures, hierarchical clustering with two clusters again gives the best score. Three internal cluster validation measures are used. Concludes that Hierarchical clustering with two clusters performs the best in (Connective, Dunn, and Silhouette). Recall that the connectivity should be minimized, while both the Dunn Index and the Silhouette Width should be maximized. Thus, Hierarchical clustering outperforms the other clustering algorithms under each validation measure.

The internalization of religious moderation values in aqidah and akhlaq courses at the Faculty of Tarbiyah and Teacher Training (FTIK) of IAIN Kerinci

The diversity of Indonesian society is reflected in the variety of ethnicities, religions, races, and social groups spread across the archipelago. Religious moderation is recognized as a key approach to addressing issues of radicalism and extremism. This study aims to gain a comprehensive understanding of the process of incorporating religious moderation values into the curriculum for students in the Islamic Education Management Department at IAIN Kerinci, Jambi Province. The research employs a descriptive qualitative approach, with data collected through observation, interviews, and documentation. The validity of the data was ensured through internal validity measures, including data triangulation, member checking, repeated observations at the research site, peer debriefing, involvement of informants in various stages of the research, and researcher bias clarification. The results reveal that the teaching style of lecturers at the Faculty of Tarbiyah and Teacher Training (FTIK) of IAIN Kerinci plays a crucial role in shaping students to become moderate, tolerant, and peace-loving individuals. Lecturers employ interactive and participatory teaching approaches, integrating religious moderation values into the curriculum using various instructional methods. Factors contributing to the internalization of these values among students include participation in extracurricular activities and campus organizations that promote interfaith dialogue, as well as involvement in social service activities with communities of different religious backgrounds.

From A-to-Z review of clustering validation indices

Data clustering involves identifying latent similarities within a dataset and organizing them into clusters or groups. The outcomes of various clustering algorithms differ as they are susceptible to the intrinsic characteristics of the original dataset, including noise and dimensionality. The effectiveness of such clustering procedures directly impacts the homogeneity of clusters, underscoring the significance of evaluating algorithmic outcomes. Consequently, the assessment of clustering quality presents a significant and complex endeavor. A pivotal aspect affecting clustering validation is the cluster validity metric, which aids in determining the optimal number of clusters. The main goal of this study is to comprehensively review and explain the mathematical operation of internal and external cluster validity indices, but not all, to categorize these indices and to brainstorm suggestions for future advancement of clustering validation research. In addition, we review and evaluate the performance of internal and external clustering validation indices on the most common clustering algorithms, such as the evolutionary clustering algorithm star (ECA*). Finally, we suggest a classification framework for examining the functionality of both internal and external clustering validation measures regarding their ideal values, user-friendliness, responsiveness to input data, and appropriateness across various fields. This classification aids researchers in selecting the appropriate clustering validation measure to suit their specific requirements.

The Adaptation of the Communicative Effectiveness Index (CETI) into Greek: A Reliability and Validity Study.

The Communicative Effectiveness Index (CETI) is an informant rating scale designed to measure changes in functional communication in people with aphasia (PWA) from the carer's perspective. It offers a comprehensive view of aphasia's impact on everyday communication situations, aiding clinicians in designing personalized intervention plans. The aim of this study was to translate and adapt the CETI into Greek (CETI-GR) and validate its psychometric properties. The CETI-GR was translated into Greek using back-translation. A pilot and a content validity study ensured its acceptability. The study involved 30 people with aphasia and 30 carers. The CETI-GR's psychometric properties were evaluated, including internal consistency, test-retest reliability, inter-rater reliability, and validity measures. The CETI-GR demonstrated excellent internal consistency (Cronbach's α > 0.95) and excellent inter-rater reliability (ICC ≥ 0.93). Excellent consistency was revealed when testing the CETI responses given only by PWA (α = 0.91) versus their carers (α = 0.97). Test-retest reliability was high (ICC = 0.88). Significant correlations between the CETI-GR and measures of language severity, functional communication, and quality of life supported convergent validity. The CETI-GR is a reliable tool for assessing functional communication in chronic aphasia. Its Greek adaptation enhances aphasia rehabilitation, enabling person-centered care and improving the quality of life for people with aphasia and carers.

Informing Structural Assumptions for Three State Oncology Cost-Effectiveness Models through Model Efficiency and Fit.

The characteristics and relative strengths and weaknesses of partitioned survival models (PSMs) and state transition models (STMs) for three state oncology cost-effectiveness models have previously been studied. Despite clear and longstanding economic modeling guidelines, more than one structure is rarely presented, and the choice of structure appears correlated more with audience or precedent than disease, decision problem, or available data. One reason may be a lack of guidance and tools available to readily compare measures of internal validity such as the model fit and efficiency of different structures, or sensitivity of results to those choices. To address this gap, methods are presented to evaluate the fit and efficiency of three structures, with an accompanying R software package, psm3mkv. The methods are illustrated by analyzing interim and final analysis datasets of the KEYNOTE-826 randomized controlled trial. At both interim and final analyses, the STM Clock Reset structure provided the best and most efficient fit. Structural uncertainties had been reduced from interim to final analysis. Beyond measures of internal validity, guidelines highlight the importance of reflecting all available data, avoiding model selection purely on the basis of goodness of fit and strongly considering external validity. The method and software allow modelers to more easily evaluate and report model fit and efficiency, examine implicit assumptions, and reveal sensitivities to structural choices.

Prediction of acute toxicity for Chlorella vulgaris caused by tire wear particle-derived compounds using quantitative structure-activity relationship models

Tire wear particles (TWPs) enter aquatic ecosystems through various pathways, such as rainwater and urban runoff. Additives in TWPs can harm aquatic organisms in these ecosystems. Therefore, it is essential to investigate their toxicity to aquatic organisms. In our study, we initially recorded the median effective concentrations of 21 TWP-derived compounds on Chlorella vulgaris growth, ranging from 0.04 to 8.60 mg/L. Subsequently, through an extensive review of the literature, we incorporated 112 compounds with specific toxicity endpoints to construct the QSAR model using genetic algorithm and multiple linear regression techniques, followed by the construction of the consensus model and the quantitative read-across structure-activity relationship (q-RASAR) model. Meanwhile, we employed rigorous internal and external validation measures to assess the performance of the model. The results indicated that the developed q-RASAR model exhibited strong adaptation, robustness, and reliable prediction, with q-RASAR indicators of Q2LOO = 0.7673, R2tr = 0.8079, R2test = 0.8610, Q2Fn = 0.8285−0.8614, and CCCtest = 0.9222. Based on an external dataset containing 128 emerging TWP-derived compounds, the model's applicability domain coverage was 90.6 %. The q-RASAR model predicted that the structure of diphenylamine was associated with higher toxicity, possibly liked to the SpMax2_Bhm and LogBCF descriptors. The established model reliably provides prediction and fills a critical data gap. These findings highlight the potential risks posed by emerging TWP-derived compounds to aquatic organisms.

Application of clustering strategy for automatic segmentation of tissue regions in mass spectrometry imaging.

Mass spectrometry imaging (MSI) has been widely used in biomedical research fields. Each pixel in MSI consists of a mass spectrum that reflects the molecule feature of the tissue spot. Because MSI contains high-dimensional datasets, it is highly desired to develop computational methods for data mining and constructing tissue segmentation maps. To visualize different tissue regions based on mass spectrum features and improve the efficiency in processing enormous data, we proposed a computational strategy that consists of four procedures including preprocessing, data reduction, clustering, and quantitative validation. In this study, we examined the combination of t-distributed stochastic neighbor embedding (t-SNE) and hierarchical clustering (HC) for MSI data analysis. Using publicly available MSI datasets, one dataset of mouse urinary bladder, and one dataset of human colorectal cancer, we demonstrated that the generated tissue segmentation maps from this combination were superior to other data reduction and clustering algorithms. Using the staining image as a reference, we assessed the performance of clustering algorithms with external and internal clustering validation measures, including purity, adjusted Rand index (ARI), Davies-Bouldin index (DBI), and spatial aggregation index (SAI). The result indicated that SAI delivered excellent performance for automatic segmentation of tissue regions in MSI. We used a clustering algorithm to construct tissue automatic segmentation in MSI datasets. The performance was evaluated by comparing it with the stained image and calculating clustering validation indexes. The results indicated that SAI is important for automatic tissue segmentation in MSI, different from traditional clustering validation measures. Compared to the reports that used internal clustering validation measures such as DBI, our method offers more effective evaluation of clustering results for MSI segmentation. We envision that the proposed automatic image segmentation strategy can facilitate deep learning in molecular feature extraction and biomarker discovery for the biomedical applications of MSI.

Assessment of the global Copernicus, NASADEM, ASTER and AW3D digital elevation models in Central and Southern Africa

ABSTRACT Validation studies of global Digital Elevation Models (DEMs) in the existing literature are limited by the diversity and spread of landscapes, terrain types considered and sparseness of groundtruth. Moreover, there are knowledge gaps on the accuracy variations in rugged and complex landscapes, and previous studies have often not relied on robust internal and external validation measures. Thus, there is still only partial understanding and limited perspective of the reliability and adequacy of global DEMs for several applications. In this study, we utilize a dense spread of LiDAR groundtruth to assess the vertical accuracies of four medium-resolution, readily available, free-access and global coverage 1 arc-second (30 m) DEMs: NASADEM, ASTER GDEM, Copernicus GLO-30, and ALOS World 3D (AW3D). The assessment is carried out at landscapes spread across Cape Town, Southern Africa (urban/industrial, agricultural, mountain, peninsula and grassland/shrubland) and forested national parks in Gabon, Central Africa (low-relief tropical rainforest and high-relief tropical rainforest). The statistical analysis is based on robust accuracy metrics that cater for normal and non-normal elevation error distribution, and error ranking. In Cape Town, Copernicus DEM generally had the least vertical error with an overall Mean Error (ME) of 0.82 m and Root Mean Square Error (RMSE) of 2.34 m while ASTER DEM had the poorest performance. However, ASTER GDEM and NASADEM performed better in the low-relief and high-relief tropical forests of Gabon. Generally, the DEM errors have a moderate to high positive correlation in forests, and a low to moderate positive correlation in mountains and urban areas. Copernicus DEM showed superior vertical accuracy in forests with less than 40% tree cover, while ASTER and NASADEM performed better in denser forests with tree cover greater than 70%. This study is a robust regional assessment of these global DEMs.

Translation and validation into Spanish of the oral health self-efficacy scale in Chilean older adults

BackgroundPopulation ageing poses a challenge to improving the well-being of older adults, particularly in terms of oral health. Promoting self-efficacy in oral health behaviours is crucial for maintaining this population’s health and quality of life. The Oral Health Self-Efficacy Scale (OHSES) has been widely used to assess dental self-efficacy and is considered comprehensive and reliable. However, there is a need to validate OHSES for Spanish-speaking older adults. This study aimed to assess the reliability and validity of the translated questionnaire for use in the older Chilean population.MethodsA sample of 188 older adults, aged 60 years and above residing independently in the community, were recruited by accessing databases from the National Senior Citizen Service and various community organisations within the region of La Araucanía. The participants underwent comprehensive oral examinations and oral health interviews, focusing on variables such as OHSES, Oral health-related quality of life (OHIP-14Sp), assessment of remaining teeth, knowledge and attitudes toward oral health, and sociodemographic characteristics. The validity of the translated questionnaire was assessed through translation and cross-cultural adaptation, cognitive debriefing, and face and content validation. The psychometric properties of the questionnaire were evaluated through measures of internal consistency (Cronbach’s alpha), content validity (Content validity index), construct validity (factor analysis and Pearson correlation analysis), and test-retest reliability (intraclass correlation).ResultsThe Spanish version of OHSES demonstrated adequate face and content validity. The confirmatory factor analysis confirmed a two-factor scale with 7 items for a better fit. The scale demonstrated high internal consistency (Cronbach’s alpha = 0.821) and acceptable test–retest reliability (ICC = 0.411). Correlations were found between the OHSES score, the number of remaining teeth, knowledge and attitudes towards oral health, and the OHIP-14Sp (p < 0.01).ConclusionsThis study confirms the validity of the Spanish version of the Oral Health Self-Efficacy Scale for older adults in Chile. The scale is expected to be helpful in assessing self-efficacy in dental interventions and collecting data for international comparisons. This research opens new dimensions in patient-reported assessment of oral health.

Clinical Assessment Interview for Negative Symptoms (CAINS): A Systematic Review of Measurement Properties.

Negative symptoms are very important for the overall loss of functioning observed in patients with schizophrenia. There is a need for valid tools to assess these symptoms. We used the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) systematic review guideline to evaluate the quality of the clinical assessment interview for negative symptoms (CAINS) as a clinician-rated outcome measurement (ClinROM). The search strategy resulted in the retrieval of 13 articles, 11 of which were included in this evaluation. In terms of risk of bias, most articles reported on measures of internal consistency and construct validity, which were overall of good quality. Structural validity, reliability, measurement error, and cross-cultural validity were reported with less than optimum quality. There was a risk of bias in ClinROM development. According to the updated criteria of good measurement properties, structural validity, internal consistency, and reliability showed good results. In contrast, hypothesis testing was somewhat poorer. Results for cross-cultural validity were indeterminate. According to the updated GRADE approach from the COSMIN group the scale received a moderate grade. The COSMIN standard allows a judgment of the CAINS as an instrument with the potential to be recommended for use, but which requires further research to assess its quality, in particular in the domains of content validity, internal consistency, and cross-cultural validity.

Improving student retention in higher education institutions – Exploring the factors influencing employees extra-role behavior

This study explores the impact of internal service quality on employee job satisfaction (JOS) and the extra role behaviors (ERB) of frontline employees (FLE) in higher education institutions (HEIs). With the hope of improving student retention, the study specifically examined how Human Resources Management Practices (HRMP), Empathetic Leadership (EML), Colleague Support (CS), and Service Technology (ST) indirectly influence FLE ERB, as well as the contextual factor of employee Job satisfaction (JOS) that affects the strength of this relationship. Drawing on social exchange theory and equity theory, the authors propose that HRMP, EML, CS, and ST have positive effects on JOS, and JOS has positive impacts on FLE ERB. The research was conducted drawing on a sample of 251 frontline employees and 251 students using the dyadic research design and self-design questionnaires as the main instrument within one HEI in the United Arab Emirates (UAE). Structural Equation Modeling (SEM) was employed utilizing SmartPLS software to perform bootstrapping routines and generate the SEM to explore the relationships between different variables. The results were analyzed using measures of internal consistency, validity, and goodness of fit. The study's findings support the proposed hypotheses except for the positive influence of job satisfaction on FLE ERB. The study highlights the mechanisms through which internal service quality can impact FLE ERB and identifies the factors that enhance this process. The results revealed that HRMP, CS, and EML have a significant positive impact on JOS, and ST has a significant positive effect on ERB. However, JOS has a significant negative impact on ERB. The study has practical implications for understanding how to promote ERB among FLE in HEIs to improve the quality of service provided to students, which can help to sustain student loyalty and retention.

Screening for post-intensive care syndrome: Validation of the Healthy Aging Brain Care Monitor Self-Report Chinese version.

The Healthy Aging Brain Care Monitor Self-Report (HABC-M SR) is a tool to evaluate physical, mental and cognitive impairments. It has been validated in several countries for post-intensive care syndrome (PICS) assessment, but there is no Chinese version yet. The aim of this study is to translate and cross-cultural adapt HABC-M SR, and validate its psychological properties among Chinese ICU survivors. This is a cross-sectional study. the HABC-M SR was translated into Chinese and validated in intensive care unit survivors (n = 145). Measures of internal consistency, construct validity, concurrent validity, and content validity were evaluated. A 19-item Chinese version of HABC-M SR was yielded, with good reliability (Cronbach's α = 0.92) and validity (the variance was 64.4%, overall content validity was 0.91, and correlation coefficients were 0.62-0.90). The 19-item Chinese version HABC-M SR is a reliable and valid tool for PICS assessing and may be regarded as a standard measurement. The Chinese version HABC-M SR may help in selecting PICS high-risk survivors for ICU follow-up interventions. The HABC-M SR can also be regarded as a standard specific PICS measurement, thus promote the comparability between studies and transformation of the clinical evidence.

97 Constant Emitting Reference Material for Material Emissions Test Procedures

Abstract In industrialised countries more than 80 % of the time is spent indoors. Products, such as building materials and furniture, emit volatile organic compounds (VOCs), which are therefore ubiquitous in indoor air. VOC in combination may, under certain environmental and occupational conditions, result in reported sensory irritation and health complaints. Emission concentrations can become further elevated in new or refurbished buildings where the rate of air exchange with fresh ambient air may be limited due to improved energy saving aspects. A healthy indoor environment can be achieved by controlling the sources and by eliminating or limiting the release of harmful substances into the air. One way is to use (building) materials proved to be low emitting. Meanwhile, a worldwide network of professional commercial and non-commercial laboratories performing emission tests for the evaluation of products for interior use has been established. Therefore, comparability of test results must be ensured. A laboratory’s proficiency can be proven by internal and external validation measures that both include the application of suitable emission reference materials (ERM). For the emission test chamber procedure according to EN 16516, no artificial ERM is commercially available. The EU-funded EMPIR project MetrIAQ aims to fill this gap by developing new and improved ERMs. The goal is to obtain a material with a reproducible and temporally constant compound release (less than 10 % variability over 14 days). Different approaches, such as the impregnation of porous materials, are being tested. The generation as well as results of the most promising materials will be presented.

Fast computation of cluster validity measures for bregman divergences and benefits

Partitional clustering is one of the most relevant unsupervised learning and pattern recognition techniques. Unfortunately, one of the main drawbacks of these methodologies refer to the fact that the number of clusters is generally assumed to be known beforehand and automating its selection is not straightforward. On the same token, internal validity measures, such as the Silhouette index, Davies-Bouldin and Caliski-Harabasz measures have emerged as the standard techniques to be used when comparing the goodness of clustering results obtained via different clustering methods. These measures take into consideration both the inter and intra-cluster simmilarities and can be adapted to different metrics. Unfortunately, their used has been hindered due to their large computational complexities, which are commonly quadratic with respect to the number of instances of the data set. In this work, we show that the time complexity of computing the most popular internal validity measures can be utterly reduced by making used of the within-cluster errors and different properties of the Bregman divergences. This contribution ultimately allows us to massively speed-up the selection of an adequate number of clusters for a given data set as verified with extensive empirical comparisons.

“Trustworthiness,” confidence in estimated effects, and confidently translating research into clinical practice

Trustworthy, preprocessed sources of evidence, such as systematic reviews and clinical practice guidelines, are crucial for practicing clinicians. Confidence in estimated effects is related to how different the outcome data were between the two groups. Factors including the effect size, variability of the effect, research integrity, research methods, and selected outcome measures impact confidence in the estimated effect. The current evidence suggests that post-randomization biases cannot be ruled out with a high degree of certainty in published research, limiting the utility of preprocessed sources for clinicians. Research should be prospectively registered to improve this situation, and fidelity with prospective intent should be verified to minimize biases and strengthen confidence in estimated effects. Otherwise, discussions related to preprocessed literature, including P-values, point estimates of effect, confidence intervals, post-randomization biases, external and internal validity measures, and the confidence in estimated effects required to translate research into practice confidently, are all moot points.

Validating the representation of distance between infarct diseases using word embedding

BackgroundThe pivot and cluster strategy (PCS) is a diagnostic reasoning strategy that automatically elicits disease clusters similar to a differential diagnosis in a batch. Although physicians know empirically which disease clusters are similar, there has been no quantitative evaluation. This study aimed to determine whether inter-disease distances between word embedding vectors using the PCS are a valid quantitative representation of similar disease groups in a limited domain.Methodss were extracted from the Ichushi Web database and subjected to morphological analysis and training using Word2Vec, FastText, and GloVe. Consequently, word embedding vectors were obtained. For words including “infarction,” we calculated the cophenetic correlation coefficient (CCC) as an internal validity measure and the adjusted rand index (ARI), normalized mutual information (NMI), and adjusted mutual information (AMI) with ICD-10 codes as the external validity measures. This was performed for each combination of metric and hierarchical clustering method.ResultsSeventy-one words included “infarction,” of which 38 diseases matched the ICD-10 standard with the appearance of 21 unique ICD-10 codes. When using Word2Vec, the CCC was most significant at 0.8690 (metric and method: euclidean and centroid), whereas the AMI was maximal at 0.4109 (metric and method: cosine and correlation, and average and weighted). The NMI and ARI were maximal at 0.8463 and 0.3593, respectively (metric and method: cosine and complete). FastText and GloVe generally resulted in the same trend as Word2Vec, and the metric and method that maximized CCC differed from the ones that maximized the external validity measures.ConclusionsThe metric and method that maximized the internal validity measure differed from those that maximized the external validity measures; both produced different results. The cosine distance should be used when considering ICD-10, and the Euclidean distance when considering the frequency of word occurrence. The distributed representation, when trained by Word2Vec on the “infarction” domain from a Japanese academic corpus, provides an objective inter-disease distance used in PCS.

Application of K-Means Clustering and Calendar View Visualisation for Air Pollution Index Analysis

Two years of diurnal concentration of particulate matter (PM10 ) and nitrogen dioxide with the addition of relative humidity measurement, collected from Putrajaya, Malaysia’s ground-based measurement station from January 2014 to December 2015, were analysed. K-means clustering was employed and optimal clusters of four were identified for each year based on the most suggested number of clusters from internal cluster validation measures of the total within sum of square, silhouette index and gap statistics. Each cluster was then profiled where each mean pollutant sub-indices were calculated and the contributing pollutant to the air pollution index (API) was determined by looking at the maximum value from all sub-indices. This mechanism closely follows the Recommended Malaysian Air Quality Guidelines (RMG) for determining API. Particulate matter was found to be the dominant sub-index in all clusters and then paired with the mean relative humidity for visualisation. A calendar view was selected to show the temporal patterns and we observed a consistent cluster profile with the actual mean values of the selected parameters for most months. The calendar view also suggested that overall, the API (based on particulate matter) in 2014 was much better as compared to 2015.

Impacts of COVID-19 on Electric Vehicle Charging Behavior: Data Analytics, Visualization, and Clustering

COVID-19 pandemic has infected millions and led to a catastrophic loss of lives globally. It has also significantly disrupted the movement of people, businesses, and industries. Additionally, electric vehicle (EV) users have faced challenges in charging their vehicles in public charging locations where there is a risk of COVID-19 exposure. However, a case study of EV charging behavior and its impacts during the SARS-CoV-2 is not addressed in the existing literature. This paper investigates the impacts of COVID-19 on EV charging behavior by analyzing the charging activity during the pandemic using a dataset from a public charging facility in the USA. Data visualization of charging behavior alongside significant timelines of the pandemic was utilized for analysis. Moreover, a cluster analysis using k-means, hierarchical clustering, and Gaussian mixture models was performed to identify common groups of charging behavior based on the vehicle arrival and departure times. Although the number of vehicles using the charging station was reduced significantly due to lockdown restrictions, the charging activity started to pick up again since May 2021 due to an increase in vaccination and easing of public restrictions. However, the charging activity currently still remains around half of the activity pre-pandemic. A noticeable decline in charging session length and an increase in energy consumption can be observed as well. Clustering algorithms identified three groups of charging behavior during the pandemic and their analysis and performance comparison using internal validation measures were also presented.

Are cluster validity measures (in) valid?

Internal cluster validity measures (such as the Calinski–Harabasz, Dunn, or Davies–Bouldin indices) are frequently used for selecting the appropriate number of partitions a dataset should be split into. In this paper we consider what happens if we treat such indices as objective functions in unsupervised learning activities. Is the optimal grouping with regards to, say, the Silhouette index really meaningful? It turns out that many cluster (in)validity indices promote clusterings that match expert knowledge quite poorly. We also introduce a new, well-performing variant of the Dunn index that is built upon OWA operators and the near-neighbour graph so that subspaces of higher density, regardless of their shapes, can be separated from each other better.

Supervised application of internal validation measures to benchmark dimensionality reduction methods in scRNA-seq data.

A typical single-cell RNA sequencing (scRNA-seq) experiment will measure on the order of 20 000 transcripts and thousands, if not millions, of cells. The high dimensionality of such data presents serious complications for traditional data analysis methods and, as such, methods to reduce dimensionality play an integral role in many analysis pipelines. However, few studies have benchmarked the performance of these methods on scRNA-seq data, with existing comparisons assessing performance via downstream analysis accuracy measures, which may confound the interpretation of their results. Here, we present the most comprehensive benchmark of dimensionality reduction methods in scRNA-seq data to date, utilizing over 300 000 compute hours to assess the performance of over 25 000 low-dimension embeddings across 33 dimensionality reduction methods and 55 scRNA-seq datasets. We employ a simple, yet novel, approach, which does not rely on the results of downstream analyses. Internal validation measures (IVMs), traditionally used as an unsupervised method to assess clustering performance, are repurposed to measure how well-formed biological clusters are after dimensionality reduction. Performance was further evaluated over nearly 200 000 000 iterations of DBSCAN, a density-based clustering algorithm, showing that hyperparameter optimization using IVMs as the objective function leads to near-optimal clustering. Methods were also assessed on the extent to which they preserve the global structure of the data, and on their computational memory and time requirements across a large range of sample sizes. Our comprehensive benchmarking analysis provides a valuable resource for researchers and aims to guide best practice for dimensionality reduction in scRNA-seq analyses, and we highlight Latent Dirichlet Allocation and Potential of Heat-diffusion for Affinity-based Transition Embedding as high-performing algorithms.