Silhouette Score Research Articles

A General Iterative Clustering Algorithm.

The quality of a cluster analysis of unlabeled units depends on the quality of the between units dissimilarity measures. Data‐dependent dissimilarity is more objective than data independent geometric measures such as Euclidean distance. As suggested by Breiman, many data driven approaches are based on decision tree ensembles, such as a random forest (RF), that produce a proximity matrix that can easily be transformed into a dissimilarity matrix. An RF can be obtained using labels that distinguish units with real data from units with synthetic data. The resulting dissimilarity matrix is input to a clustering program and units are assigned labels corresponding to cluster membership. We introduce a general iterative cluster (GIC) algorithm that improves the proximity matrix and clusters of the base RF. The cluster labels are used to grow a new RF yielding an updated proximity matrix, which is entered into the clustering program. The process is repeated until convergence. The same procedure can be used with many base procedures such as the extremely randomized tree ensemble. We evaluate the performance of the GIC algorithm using benchmark and simulated data sets. The properties measured by the Silhouette score are substantially superior to the base clustering algorithm. The GIC package has been released in R: https://cran.r‐project.org/web/packages/GIC/index.html.

The Effect of Text Summarization in Essay Scoring (Case Study: Teach on E-Learning)

The development of automated essay scoring (AES) in the neural network (NN) approach has eliminated feature engineering. However, feature engineering is still needed, moreover, data with labels in the form of rubric scores, which are complementary to AES holistic scores, are still rarely found. In general, data without labels/scores is found more. However, unsupervised AES research has not progressed with the more common use of publicly labeled data. Based on the case studies adopted in the research, automatic text summarization (ATS) was used as a feature engineering model of AES and readability index as the definition of rubric values for data without labels.This research focuses on developing AES by implementing ATS results on SOM and HDBSCAN. The data used in this research are 403 documents of TEACH ON E-learning essays. Data is represented in the form of a combination of word vectors and a readability index. Based on the tests and measurements carried out, it was concluded that AES with ATS implementation had no good potential for the assessment of TEACH ON essays in increasing the silhouette score. The model produces the best silhouette score of 0.727286113 with original essay data.

Group-tailored feedback on online mental health screening for university students: using cluster analysis

BackgroundThe method by which mental health screening result reports are given affects the user’s health behavior. Lists with the distribution of scores in various mental health areas is difficult for users to understand, and if the results are negative, they may feel more embarrassed than necessary. Therefore, we propose using group-tailored feedback, grouping people of similar mental health types by cluster analysis for comprehensive explanations of multidimensional mental health.MethodsThis cross-sectional, observational study was conducted using a qualitative approach based on cluster analysis. Data were collected via a developed mental screening website, with depression, anxiety, sleep problems, perfectionism, procrastination, and attention assessed for 2 weeks in January 2020 in Korea. Participants were randomly recruited, and sample size was 174. Total was divided into 25 with severe depression/anxiety (SDA+) and 149 without severe depression/anxiety (SDA-) according to the PHQ-9 and GAD-7 criteria. Cluster analysis was conducted in each group, and an ANOVA was performed to find significant clusters. Thereafter, structured discussion was performed with mental health professionals to define the features of the clusters and construct the feedback content initially. Thirteen expert counselors were interviewed to reconstruct the content and validate the effectiveness of the developed feedback.ResultsSDA- was divided into 3 using the k-means algorithm, which showed the best performance (silhouette score = 0.32, CH score = 91.67) among the clustering methods. Perfectionism and procrastination were significant factors in discretizing the groups. SDA+ subgroups were integrated because only 25 people belonged to this group, and they need professional help rather than self-care. Mental status and treatment recommendations were determined for each group, and group names were assigned to represent their features. The developed feedback was assessed to improve mental health literacy (MHL) through integrative and understandable explanations of multidimensional mental health. Moreover, it appeared that a sense of belonging was induced to reduce reluctance to face the feedback.ConclusionsThis study suggests group-tailored feedback using cluster analysis, which identifies groups of university students by integrating multidimensions of mental health. These methods can help students increase their interest in mental health and improve MHL to enable timely help.

A new non-archimedean metric on persistent homology

In this article, we define a new non-archimedean metric structure, called cophenetic metric, on persistent homology classes of all degrees. We then show that zeroth persistent homology together with the cophenetic metric and hierarchical clustering algorithms with a number of different metrics do deliver statistically verifiable commensurate topological information based on experimental results we obtained on different datasets. We also observe that the resulting clusters coming from cophenetic distance do shine in terms of different evaluation measures such as silhouette score and the Rand index. Moreover, since the cophenetic metric is defined for all homology degrees, one can now display the inter-relations of persistent homology classes in all degrees via rooted trees.

Customer Mobile Behavioral Segmentation and Analysis in Telecom Using Machine Learning

ABSTRACT This study aims to identify telecom customer segments by utilizing machine learning and subsequently develop a web-based dashboard. The dashboard visualizes the cluster analysis based on demographics, behavior, and region features. The study applied analytic pipeline that involved five stages i.e. data generation, data pre-processing, data clustering, clusters analysis, and data visualization. Firstly, the customer’s dataset was generated using Faker Python package. Secondly was the pre-processing which includes the dimensionality reduction of the dataset using the PCA technique and finding the optimal number of clusters using the Elbow method. Unsupervised machine learning algorithm K-means was used to cluster the data, and these results were analyzed and labeled with labels and descriptions. Lastly, a dashboard was developed using Microsoft Power BI to visualize the clustering results in meaningful analysis. According to the results, four customer clusters were obtained. An interactive web-based dashboard called INSIGHT was developed to provide analysis of customer segments based on demographic, behavioral, and regional traits; and to devise customized query for deeper analysis. The correctness of the clustering results was evaluated and achieved a satisfactory Silhouette Score of 0.3853. Hence, the telecom could target their customers accurately based on their needs and preferences to increase service satisfaction.

A Novel Hybrid Clustering Approach Based on Black Hole Algorithm for Document Clustering

In information retrieval and text mining, document clustering is a big challenge because the amount of document collection has been increasing, day by day. The problem of clustering is NP-hard, use of meta-heuristic algorithms to solve these problems could be an effective method. When the solution space is large, traditional methods are unable to find a solution in a reasonable amount of time. K-means is a heuristic clustering algorithm, two main issues with heuristic algorithms are the early convergence and trapping in local optima. Moreover, finding the right number of clusters is one of the main drawbacks of the k-means algorithm. The correct value of k is always confusing, different researchers used different methods to solve this problem. To overcome these mentioned problems, this study presents a novel Hybrid approach for document clustering. One of the challenges in existing BH algorithm is the input data type. Recently, the algorithm was only accepting textual data. Another flaw in the existing model is that it doesn’t choose how many clusters k to form automatically, and the centroids are chosen at random in it. In this paper, we have constructed a Hybrid cluster identification approach which consists of the Elbow method and Silhouette score for cluster k identification. This paper mainly offers three novel combination of model to represent text documents, namely i) K-mean++ - BH + TF-IDF with fix k ii) K-mean++ - BH + W2V with fix k iii) Hybrid Black Hole with automated k. The proposed improvements have validated on the document clustering problem. Cluster analysis based on two evaluation measures, external (Purity) and internal measures (Silhouette score) are used to report the findings. Experiments have been carried out on the four al-phanumeric datasets (Doc50, Reuters, WebKB and News20) as well as on two numeric datasets (Iris and Wine) respectively. The complete result analysis is reported in detail with respect to each research contribution to compare the performance of the proposed algorithm with existing clustering methods. Result shows that the proposed Hybrid BH algorithm outperforms better than the existing clustering methods for all datasets. The clustering of data with and without stop words is examined; additionally, the two alternative word embedding used for data exploration in conjunction with proposed model are also evaluated. In the present study, proposed Hybrid BH algorithm handles the optimal value of k efficiently. This is one of the major contributions of the paper, concluded that Hybrid Black Hole is an effective algorithm for cluster analysis.

CLUSTERING VIETNAMESE CONVERSATIONS FROM FACEBOOK PAGE TO BUILD TRAINING DATASET FOR CHATBOT

The biggest challenge of building chatbots is training data. The required data must be realistic and large enough to train chatbots. We create a tool to get actual training data from Facebook messenger of a Facebook page. After text preprocessing steps, the newly obtained dataset generates FVnC and Sample dataset. We use the Retraining of BERT for Vietnamese (PhoBERT) to extract features of our text data. K-Means and DBSCAN clustering algorithms are used for clustering tasks based on output embeddings from PhoBERT$_{base}$. We apply V-measure score and Silhouette score to evaluate the performance of clustering algorithms. We also demonstrate the efficiency of PhoBERT compared to other models in feature extraction on the Sample dataset and wiki dataset. A GridSearch algorithm that combines both clustering evaluations is also proposed to find optimal parameters. Thanks to clustering such a number of conversations, we save a lot of time and effort to build data and storylines for training chatbot.

Anomaly Detection Method of Aircraft System using Multivariate Time Series Clustering and Classification Techniques

The paper presents an anomaly detection method that identifies and explains anomalies in an aircraft system based on explainable multivariate time series clustering techniques. The method considers the cyclicity of each variable within the flight phases and selects those behind the anomalies. It combines the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) and a modified Dynamic Time Warping (DTW) distance algorithms to detect abnormal behavioral profiles within the collected flight phases without any prior knowledge on the system's behavior. The proposed method explains those abnormal profiles compared to normal profiles using a new importance score. Profiles are detected using the Time Series Forest (TSF) and the silhouette criterion. The method is trained and tested using a sample from the Bombardier's Aircraft Health Monitoring System. It distinguishes the normal and abnormal behaviors by achieving a clustering silhouette score of 0.95 and detects unknown profiles with a precision of 89%.

Decision-making in formation of mean-VaR optimal portfolio by selecting stocks using K-means and average linkage clustering

Stock is one of the investment assets that has its charm for investors. It is very liquid and has a high rate of return, but it has a high risk. The strategy commonly used to minimize investment risk is to diversify through portfolio formation. A good allocation of funds must be determined in forming an optimal portfolio. In addition, the method of stock selection needs to be considered so the stocks are well diversified and the portfolio developed has good performance. This study aims to compare stock selection between K-Means and Average Linkage clustering approaches in forming an investment portfolio. Clustering analysis is used to group IDX80 stocks based on their attributes. In forming a portfolio with the Mean-VaR model, the stock selection decision criteria used are by selecting stocks with the highest positive returns from each cluster. As a result, the two clustering techniques show the superiority of the Silhouette score for a certain number of clusters, but there are still more advantages in Average Linkage. The portfolio approached by Average Linkage resulted in a better performance than the portfolio approached by K-Means. Therefore, Average Linkage clustering can be used as a better recommendation in decision-making to select stocks so as to produce optimal portfolio performance.

Analysis and modelling of a ML-based NIDS for IoT networks

As cyber-attacks become increasingly sophisticated, cybersecurity threats continue to increase. Therefore, it becomes a great challenge to detect, identify and prevent adversary attacks. The inability to detect and avert such attacks can break down the credibility of security services, e.g. data confidentiality, availability, and integrity. An intrusion detection system aims to detect, identify and respond to malicious activities involving computing and network resources. This is even more difficult when the network contains Internet of Things (IoT) devices. The process of developing an intrusion detection system is a tremendous challenge due to false alarm rates, low detection rates, unbalanced datasets and response time. This paper gives a comprehensive model of a network intrusion detection system (NIDS) by applying supervised and unsupervised machine learning (ML) over the NF-ToN-IoT-v2 dataset. Using supervised learning, implemented by Azure automated ML (AML), it is shown that the best results are achieved by algorithm XGBoostClassifier, obtaining an F-Score of 98.8%. When a custom made automated ML (AE2EML) is implemented, the best results are achieved by Random Forest Classifier, obtaining an F-Score of 98.6%. Using clustering with PCA (Principal Component Analysis), implemented by PyCaret automated ML, the proposed ML-based NIDS achieves a Silhouette score of 0.553, Calinski-Harabasz index of 1533106 and Davies-Bouldin index of 0.631.

Multivariate Wind Turbine Power Curve Model Based on Data Clustering and Polynomial LASSO Regression

Wind turbine performance monitoring is a complex task because of the non-stationary operation conditions and because the power has a multivariate dependence on the ambient conditions and working parameters. This motivates the research about the use of SCADA data for constructing reliable models applicable in wind turbine performance monitoring. The present work is devoted to multivariate wind turbine power curves, which can be conceived of as multiple input, single output models. The output is the power of the target wind turbine, and the input variables are the wind speed and additional covariates, which in this work are the blade pitch and rotor speed. The objective of this study is to contribute to the formulation of multivariate wind turbine power curve models, which conjugate precision and simplicity and are therefore appropriate for industrial applications. The non-linearity of the relation between the input variables and the output was taken into account through the simplification of a polynomial LASSO regression: the advantages of this are that the input variables selection is performed automatically. The k-means algorithm was employed for automatic multi-dimensional data clustering, and a separate sub-model was formulated for each cluster, whose total number was selected by analyzing the silhouette score. The proposed method was tested on the SCADA data of an industrial Vestas V52 wind turbine. It resulted that the most appropriate number of clusters was three, which fairly resembles the main features of the wind turbine control. As expected, the importance of the different input variables varied with the cluster. The achieved model validation error metrics are the following: the mean absolute percentage error was in the order of 7.2%, and the average difference of mean percentage errors on random subsets of the target data set was of the order of 0.001%. This indicates that the proposed model, despite its simplicity, can be reliably employed for wind turbine power monitoring and for evaluating accumulated performance changes due to aging and/or optimization.

Identifying multimodal imaging‐driven subtypes in mild cognitive impairment using deep multiview learning

AbstractBackgroundMultimodal neuroimaging data can provide complementary information that a single modality cannot about neurodegenerative diseases such as Alzheimer's disease (AD). Deep Generalized Canonical Correlation Analysis (DGCCA) is able to learn a shared feature representation from different views of data by applying non‐linear transformation using neural network. We utilize DGCCA to extract maximally correlated components from multi‐modal neuroimaging data to identify potential imaging‐driven MCI subtypes.MethodWe study 308 Mild Cognitive Impairment (MCI) participants (195 early MCI and 113 late MCI) from the Alzheimer’s Disease Neuroimaging Initiative (ADNI), each with voxel level features from FDG PET, amyloid PET (AV45) and structural MRI processed using voxel‐based morphometry (VBM). Six experimental settings are designed to compare single modality with multiview methods ‐ GCCA and DGCCA, see Figure 1. Agglomerative clustering was used to generated 2 clusters with features from each experiment. To investigate differences between the clusters, Wilcoxon rank‐sum tests are conducted on 11 baseline AD biomarkers including 5 cognitive assessments and 6 brain volume measures, from the ADNI QT‐PAD dataset http://www.pi4cs.org/qt‐pad‐challenge.ResultAmong the two multiview methods, DGCCA is able to explain 68.57% variance with 20 features, while GCCA explains 68.66% variance with 94 features. To evaluate the potential subtypes from clustering, the Calinski‐Harabasz (CH) score, Silhouette score and adjusted mutual information (AMI) score are computed, see Table 1. AV45 generates the best defined clusters, where DGCCA generates clusters with quality comparable to single modality features. In our QT analysis, clusters from FDG and DGCCA features show differential measure in all biomarkers where DGCCA learns from multimodal data, see Figure 2.ConclusionDGCCA is able to learn maximally correlated features from multimodal neuroimaging data with reduced dimensionality, and explain more variance than its linear counterpart GCCA. Cluster analysis shows these imaging‐driven MCI subtypes are different from the currently diagnosis with differential QT measures, by incorporating complementary information from 3 imaging modalities. DGCCA shows to be an effective feature learning method, and this multiview learning framework can identify potentially novel MCI subtypes to facilitate early detection of AD.

Accumulation of depressive symptoms over time in cognitively impaired individuals

AbstractBackgroundDepressive symptoms in elderly are common, but the persistence of depressive symptoms in older adults with and without cognitive impairment (CI) is less studied. Accumulation of depressive symptoms over time might be a sign of developing chronic depressive course and therefore increased risk for cognitive decline and dementia. The aim of the present study was to assess patterns of depressive symptom accumulation over time among older adults, and to explore the association between the course of depressive symptoms over time and demographic and clinical characteristics at baseline.MethodA data‐driven approach was used to cluster ADNI participants based of their longitudinal Geriatric Depression Scores (GDS‐15). We leveraged modern time‐series analysis techniques including the Dynamic Time Warping and Dynamic Barycenter Averaging as the clustering generation technique. ADNI participants were clustered to data‐driven groups presenting with unique time course of depressive symptom accumulation.ResultWe ran a search for the optimal number of clusters using the silhouette score as the indicator of a strong clustering. This resulted in the generation of 3 unique profiles with differing patient demographics. Specifically, 1st group presented with the lowest baseline depressive symptoms (GDS=0.44±0.61) and stayed stable over the course of ADNI study (GDS=1.03±1.25). P2nd group presented with slightly elevated depressive symptoms at baseline (GDS=1.78±1.10) and accumulated depressive symptoms (GDS=2.74±2.05). In contrast, 3rd group presented with significant depressive symptoms (GDS=3.80±1.42) and accumulated more symptoms over time (GDS=4.50±2.56). Although groups did not different in age at the baseline or prevalence of females, patients that have CI, indicated by both the clinical diagnosis and the patient's Clinical Dementia Rating, are more likely to have higher GDS scores and are at higher risk of developing depressive symptoms (i.e., baseline CI prevalence of 40% in Group 1 compared to 74% and 75% in Groups 2 and 3, respectively).ConclusionThe data shows that there are differences in the persistence of the depressive symptoms in elderly population with and without CI. Identifying individuals that are more susceptible to depressive symptom accumulation might significantly improved patient outcomes and higher quality of life.

Abstract 11946: Comparison of Unsupervised Learning Approaches Applied to Electronic Health Record Traits in Heart Failure

Introduction: Unsupervised machine learning (UML) applied to high dimensional data has been used to discover cardiovascular disease subtypes; however, the reproducibility of subtypes identified by different algorithms has not been explored. We compared the ability of several promising UML and clustering algorithms to identify heart failure (HF) subtypes using high dimensional electronic health record (EHR) data. Methods: Using the Penn Medicine EHR, we identified all patients who had > 2 instances of ICD-10-CM HF diagnosis. We extracted 1272 EHR-based features (vital signs, demographics, echocardiographic measurements, laboratories, comorbidities) from time of HF diagnosis and limited the cohort based on data completeness (n=8569). We selected the following methods based on prior success in simulation studies and used them to identify HF subtypes: Similarity Network Fusion (SNF), Locally Linear Embedding (LLE), Modified LLE, Uniform Manifold Approximation and Projection (UMAP), and Principal Component Analysis (PCA) followed by several clustering algorithms including K-means, Density-based spatial clustering of applications with noise (DBSCAN), and Spectral Clustering. K groups 2-12 were evaluated. Clustering performance was assessed by silhouette score and visual separation. Results: Model visualizations are shown in the Figure. Highest silhouette score achieved for each model varied widely from 0.02-0.62; optimal cluster number ranged from 2-4 across models. Normalization and standardization of continuous data did not significantly alter silhouette scores or optimal cluster number. Conclusions: HF subtypes identified through UML applied to EHR data may vary substantially depending on the algorithms used. Benchmarking strategies to evaluate reproducibility of UML in the EHR are needed to ensure valid HF patient stratification and phenotypic refinement.

Data Mining for Determining The Best Cluster Of Student Instagram Account As New Student Admission Influencer

Purpose: This study aims to apply the web data extraction method to extract student Instagram account data and the K-Means data mining method to perform clustering automatically to determine the best cluster of students' Instagram accounts as influencers for new student admissions.Design/methodology/approach: This study implemented the web data extraction method to extract student Instagram account data. This study also implemented a data mining method called K-Means to cluster data and the Silhouette Coefficient method to determine the best number of clusters.Findings/result: This study has succeeded in determining the seven best student accounts from 100 accounts that can be used as influencers for new student admissions with the highest Silhouette Score for the number of influencers selected between 5-10, which is 0.608 of the 22 clusters.Originality/value/state of the art: Research related to the determination of the best cluster of students' Instagram accounts as new student admissions influencers using web data extraction and K-Means has never been done in previous studies.

Feature selection for unsupervised machine learning of accelerometer data physical activity clusters - A systematic review.

Identifying clusters of physical activity (PA) from accelerometer data is important to identify levels of sedentary behaviour and physical activity associated with risks of serious health conditions and time spent engaging in healthy PA. Unsupervised machine learning models can capture PA in everyday free-living activity without the need for labelled data. However, there is scant research addressing the selection of features from accelerometer data. The aim of this systematic review is to summarise feature selection techniques applied in studies concerned with unsupervised machine learning of accelerometer-based device obtained physical activity, and to identify commonly used features identified through these techniques. Feature selection methods can reduce the complexity and computational burden of these models by removing less important features and assist in understanding the relative importance of feature sets and individual features in clustering. We conducted a systematic search of Pubmed, Medline, Google Scholar, Scopus, Arxiv and Web of Science databases to identify studies published before January 2021 which used feature selection methods to derive PA clusters using unsupervised machine learning models. A total of 13 studies were eligible for inclusion within the review. The most popular feature selection techniques were Principal Component Analysis (PCA) and correlation-based methods, with k-means frequently used in clustering accelerometer data. Cluster quality evaluation methods were diverse, including both external (e.g. cluster purity) or internal evaluation measures (silhouette score most frequently). Only four of the 13 studies had more than 25 participants and only four studies included two or more datasets. There is a need to assess multiple feature selection methods upon large cohort data consisting of multiple (3 or more) PA datasets. The cut-off criteria e.g. number of components, pairwise correlation value, explained variance ratio for PCA, etc. should be expressly stated along with any hyperparameters used in clustering.

Machine Learning for Subtyping Concussion Using a Clustering Approach

Background: Concussion subtypes are typically organized into commonly affected symptom areas or a combination of affected systems, an approach that may be flawed by bias in conceptualization or the inherent limitations of interdisciplinary expertise.Objective: The purpose of this study was to determine whether a bottom-up, unsupervised, machine learning approach, could more accurately support concussion subtyping.Methods: Initial patient intake data as well as objective outcome measures including, the Patient-Reported Outcomes Measurement Information System (PROMIS), Dizziness Handicap Inventory (DHI), Pain Catastrophizing Scale (PCS), and Immediate Post-Concussion Assessment and Cognitive Testing Tool (ImPACT) were retrospectively extracted from the Advance Concussion Clinic's database. A correlation matrix and principal component analysis (PCA) were used to reduce the dimensionality of the dataset. Sklearn's agglomerative clustering algorithm was then applied, and the optimal number of clusters within the patient database were generated. Between-group comparisons among the formed clusters were performed using a Mann-Whitney U test.Results: Two hundred seventy-five patients within the clinics database were analyzed. Five distinct clusters emerged from the data when maximizing the Silhouette score (0.36) and minimizing the Davies-Bouldin score (0.83). Concussion subtypes derived demonstrated clinically distinct profiles, with statistically significant differences (p < 0.05) between all five clusters.Conclusion: This machine learning approach enabled the identification and characterization of five distinct concussion subtypes, which were best understood according to levels of complexity, ranging from Extremely Complex to Minimally Complex. Understanding concussion in terms of Complexity with the utilization of artificial intelligence, could provide a more accurate concussion classification or subtype approach; one that better reflects the true heterogeneity and complex system disruptions associated with mild traumatic brain injury.

Deep significance clustering: a novel approach for identifying risk-stratified and predictive patient subgroups.

ObjectiveDeep significance clustering (DICE) is a self-supervised learning framework. DICE identifies clinically similar and risk-stratified subgroups that neither unsupervised clustering algorithms nor supervised risk prediction algorithms alone are guaranteed to generate.Materials and MethodsEnabled by an optimization process that enforces statistical significance between the outcome and subgroup membership, DICE jointly trains 3 components, representation learning, clustering, and outcome prediction while providing interpretability to the deep representations. DICE also allows unseen patients to be predicted into trained subgroups for population-level risk stratification. We evaluated DICE using electronic health record datasets derived from 2 urban hospitals. Outcomes and patient cohorts used include discharge disposition to home among heart failure (HF) patients and acute kidney injury among COVID-19 (Cov-AKI) patients, respectively.ResultsCompared to baseline approaches including principal component analysis, DICE demonstrated superior performance in the cluster purity metrics: Silhouette score (0.48 for HF, 0.51 for Cov-AKI), Calinski-Harabasz index (212 for HF, 254 for Cov-AKI), and Davies-Bouldin index (0.86 for HF, 0.66 for Cov-AKI), and prediction metric: area under the Receiver operating characteristic (ROC) curve (0.83 for HF, 0.78 for Cov-AKI). Clinical evaluation of DICE-generated subgroups revealed more meaningful distributions of member characteristics across subgroups, and higher risk ratios between subgroups. Furthermore, DICE-generated subgroup membership alone was moderately predictive of outcomes.DiscussionDICE addresses a gap in current machine learning approaches where predicted risk may not lead directly to actionable clinical steps.ConclusionDICE demonstrated the potential to apply in heterogeneous populations, where having the same quantitative risk does not equate with having a similar clinical profile.

Unsupervised methods in LC-MS data treatment: Application for potential chemotaxonomic markers search

The combination of Liquid Chromatography and Mass Spectrometry (LC-MS) is commonly used to determine and characterize biologically active compounds because of its high resolution and sensitivity. In this work we explore the interpretation of LC-MS data using multivariate statistical analysis algorithms to extract useful chemical information and identify clusters of similar samples. Samples of leaves from 19 plants belonging to the Apiaceae family were analyzed in unified LC conditions by high- and low-resolution mass spectrometry in a wide range scan mode. LC-MS data preprocessing was performed followed by statistical analysis using tensor decomposition in the form of Parallel Factor Analysis (PARAFAC); matrix factorization following tensor unfolding with principal component analysis (PCA), independent component analysis (ICA), non-negative matrix factorization (NMF); or unsupervised feature selection (UFS). The optimal number of components for each of these methods were found and results were compared using four different metrics: silhouette score, Davies-Bouldin index, computational time, number of noisy components. It was found that PCA, ICA and UFS give the best results across the majority of the criteria for both low- and high-resolution data. An algorithm for biomarker signal selection is suggested and 23 potential chemotaxonomic markers were tentatively identified using MS2 data. Dendrograms constructed by the methods were compared to the molecular phylogenic tree by calculating pixel-wise mean square error (MSE). Therefore, the suggested approach can support chemotaxonomic studies and yield valuable chemical information for biomarker discovery.

Subseasonal-to-Seasonal Extreme Precipitation Events in the Contiguous United States: Generation of a Database and Climatology

AbstractExtreme precipitation across multiple time scales is a natural hazard that creates a significant risk to life, with a commensurately large cost through property loss. We devise a method to create 14-day extreme-event windows that characterize precipitation events in the contiguous United States (CONUS) for the years 1915–2018. Our algorithm imposes thresholds for both total precipitation and the duration of the precipitation to identify events with sufficient length to accentuate the synoptic and longer time scale contribution to the precipitation event. Kernel density estimation is employed to create extreme-event polygons that are formed into a database spanning from 1915 through 2018. Using the developed database, we clustered events into regions using a k-means algorithm. We define the “hybrid index,” a weighted composite of silhouette score and number of clustered events, to show that the optimal number of clusters is 15. We also show that 14-day extreme precipitation events are increasing in the CONUS, specifically in the Dakotas and much of New England. The algorithm presented in this work is designed to be sufficiently flexible to be extended to any desired number of days on the subseasonal-to-seasonal (S2S) time scale (e.g., 30 days). Additional databases generated using this framework are available for download from our GitHub. Consequently, these S2S databases can be analyzed in future works to determine the climatology of S2S extreme precipitation events and be used for predictability studies for identified events.