Distance Weighted Discrimination Research Articles

Asymptotic behavior of some multicategory classification methods for high-dimensional data

We consider multicategory extensions of binary discrimination methods via one-versus-one (OVO) or one-versus-rest (OVR) methodologies, focusing on extensions of the binary classification by linear mean difference (MD), support vector machine (SVM), maximal data piling (MDP), and distance weighted discrimination (DWD) via OVO, and the multicategory extension of MD via OVR, in the context of high-dimensional and low sample size (HDLSS) data. The asymptotic behavior of OVO-MD, OVO-SVM, OVO-MDP and OVO-DWD is described when the dimension of the data increases and the sample size is fixed, in terms of the probabilities of correct classification of a new data point, finding sufficient conditions for the correct classification probabilities to converge to one as the dimension approaches infinity. As in the binary case, OVO-MD, OVO-SVM and OVO-MDP have the same asymptotic behavior while OVO-DWD could behave differently. We also consider the asymptotic behavior of the OVR-MD methodology providing necessary and sufficient conditions for a new data point of a given class to be correctly classified with probability tending to one. A simulation experiment is conducted to further compare the methodologies, and consider the four binary methods in the OVR case. We evaluate the performance of the considered methods using a microarray data set.

Improvement of Classification Performance in High-Dimension Low-Sample-Size Modeling by Sparse Functional Connectivity States in Subjects with Attention Deficit-Hyperactivity Disorder and Healthy Controls

Background: The precise identification of attention deficit-hyperactivity disorder (ADHD) is one of the challenging clinical processes. Disorganizations in functional neural networks revealed via functional magnetic resonance imaging have recently been contributing. Machine learning approaches, particularly classification methods, have commonly been employed as a framework for diverse data analysis, indicating promising medical diagnosis results. However, as the neuroimaging data are high-dimensional with a low sample size (the current dataset), this study aimed to evaluate the classification performance of the models by considering the specific contribution of the sparsity of data matrices. Methods: This cross-sectional study analyzed the preprocessed data from the 2011 ADHD-200 Global Competition. A total of 768 and 171 data items were considered training and test, respectively. The diagnosis status was used as a response variable. Age, gender, hand dominance, and activity relationship between 116 brain regions derived from inverse covariance matrix and inverse sparse covariance matrix were used as predictive variables. Accordingly, this study compared the performance of three models, namely support vector machine (SVM), distance-weighted discrimination (DWD), and data maximum dispersion classifier (DMDC) for ADHD categorization. Results: The highest value for the total accuracy was reported for the SVM model on the sparse covariance matrix. Moreover, the highest values for the balanced classification rate (BCR) (59%) and sensitivity (64%) were reported for DMDC on the sparse covariance matrix. The best level of specificity (99%) was obtained from DWD using the sparse covariance matrix. The highest levels of the values (i.e., total accuracy and BCR) were achieved through the model fitting on the sparse matrices. Among the six models, the DMDC model on sparse covariance matrix was the most optimal algorithm due to the superiority of the two indices (i.e., accuracy: 60% and BCR: 60%) and the favorable balance between sensitivity and specificity values. Conclusions: Among the current studied three models, DMDC performance, applying the sparse data, remarkably improved the results of classification processes. Based on the present findings, the neuronal connectivity among subcortical structures comprising parts of the basal ganglia and cerebellum provides a distinction between ADHD subjects and healthy controls.

Distance‐weighted discrimination for functional data

The main contribution of the paper is the development of a new margin‐based classifier called distance‐weighted discrimination (DWD) for functional data classification. The proposed classifier employs functional principal component analysis (FPCA) to reduce the dimensionality of the functional data and is free of the restrictive assumptions imposed by Bayes classifiers in terms of mean and covariance functions. Theoretical results show that the proposed classifier is Bayes risk consistent under mild assumptions. Simulation studies and real data examples demonstrate that the DWD classifier outperforms several conventional classifiers in terms of prediction accuracy. Overall, the paper provides a new approach for functional data classification with good empirical performance.

Lung proteome and metabolome endotype in HIV-associated obstructive lung disease.

Obstructive lung disease is increasingly common among persons with HIV, both smokers and nonsmokers. We used aptamer proteomics to identify proteins and associated pathways in HIV-associated obstructive lung disease. Bronchoalveolar lavage fluid (BALF) samples from 26 persons living with HIV with obstructive lung disease were matched to persons living with HIV without obstructive lung disease based on age, smoking status and antiretroviral treatment. 6414 proteins were measured using SomaScan® aptamer-based assay. We used sparse distance-weighted discrimination (sDWD) to test for a difference in protein expression and permutation tests to identify univariate associations between proteins and forced expiratory volume in 1 s % predicted (FEV1 % pred). Significant proteins were entered into a pathway over-representation analysis. We also constructed protein-driven endotypes using K-means clustering and performed over-representation analysis on the proteins that were significantly different between clusters. We compared protein-associated clusters to those obtained from BALF and plasma metabolomics data on the same patient cohort. After filtering, we retained 3872 proteins for further analysis. Based on sDWD, protein expression was able to separate cases and controls. We found 575 proteins that were significantly correlated with FEV1 % pred after multiple comparisons adjustment. We identified two protein-driven endotypes, one of which was associated with poor lung function, and found that insulin and apoptosis pathways were differentially represented. We found similar clusters driven by metabolomics in BALF but not plasma. Protein expression differs in persons living with HIV with and without obstructive lung disease. We were not able to identify specific pathways differentially expressed among patients based on FEV1 % pred; however, we identified a unique protein endotype associated with insulin and apoptotic pathways.

Lung and Plasma Metabolome in HIV-Associated Obstructive Lung Disease.

HIV is a risk factor for obstructive lung disease (OLD), independent of smoking. We used mass spectrometry (MS) approaches to identify metabolomic biomarkers that inform mechanistic pathogenesis of OLD in persons with HIV (PWH). We obtained bronchoalveolar lavage fluid (BALF) samples from 52 PWH, in case:control (+OLD/-OLD) pairs matched on age, smoking status, and antiretroviral treatment. Four hundred nine metabolites from 8 families were measured on BALF and plasma samples using a MS-based Biocrates platform. After filtering metabolites with a high proportion of missing values and values below the level of detection, we performed univariate testing using paired t tests followed by false discovery rate corrections. We used distance-weighted discrimination (DWD) to test for an overall difference in the metabolite profile between cases and controls. After filtering, there were 252 BALF metabolites for analysis from 8 metabolite families. DWD testing found that collectively, BALF metabolites differentiated cases from controls, whereas plasma metabolites did not. In BALF samples, we identified 3 metabolites that correlated with OLD at the false discovery rate of 10%; all were in the phosphatidylcholine family. We identified additional BALF metabolites when analyzing lung function as a continuous variable, and these included acylcarnitines, triglycerides, and a cholesterol ester. Collectively, BALF metabolites differentiate PWH with and without OLD. These included several BALF lipid metabolites. These findings were limited to BALF and were not found in plasma from the same individuals. Phosphatidylcholine, the most common lipid component of surfactant, was the predominant lipid metabolite differentially expressed.

Maximum Decentral Projection Margin Classifier for High Dimension and Low Sample Size problems

Compared with relatively easy feature creation or generation in data analysis, manual data labeling needs a lot of time and effort in most cases. Even if automated data labeling​ seems to make it better in some cases, the labeling results still need to be checked and verified by manual. The High Dimension and Low Sample Size (HDLSS) data are therefore very common in data mining and machine learning. For classification problems with the HDLSS data, due to data piling and approximate equidistance between any two input points in high-dimension space, some traditional classifiers often give poor predictive performance. In this paper, we propose a Maximum Decentral Projection Margin Classifier (MDPMC) in the framework of a Support Vector Classifier (SVC). In the MDPMC model, the constraints of maximizing the projection distance between decentralized input points and their supporting hyperplane are integrated into the SVC model in addition to maximizing the margin of two supporting hyperplanes. On ten real HDLSS datasets, the experiment results show that the proposed MDPMC approach can deal well with data piling and approximate equidistance problems. Compared with SVC with Linear Kernel (SVC-LK) and Radial Basis Function Kernel (SVC-RBFK), Distance Weighted Discrimination (DWD), weighted DWD (wDWD), Distance-Weighted Support Vector Machine (DWSVM), Population-Guided Large Margin Classifier (PGLMC), and Data Maximum Dispersion Classifier (DMDC), MDPMC obtains better predictive accuracy and lower classification errors than the other seven classifiers on the HDLSS data.

A Reproducing Kernel Hilbert Space Framework for Functional Classification

The intrinsic infinite-dimensional nature of functional data creates a bottleneck in the application of traditional classifiers to functional settings. These classifiers are generally either unable to generalize to infinite dimensions or have poor performance due to the curse of dimensionality. To address this concern, we propose building a distance-weighted discrimination (DWD) classifier on scores obtained by projecting data onto one specific direction. We choose this direction by minimizing, over a reproducing kernel Hilbert space, an empirical risk function containing the DWD classifier loss function. Our proposed classifier avoids overfitting and enjoys the appealing properties of DWD classifiers. We further extend this framework to accommodate functional data classification problems where scalar covariates are involved. In contrast to previous work, we establish a nonasymptotic estimation error bound on the relative misclassification rate. Through simulation studies and a real-world application, we demonstrate that the proposed classifier performs favorably relative to other commonly used functional classifiers in terms of prediction accuracy in finite-sample settings. Supplementary materials for this article are available online.

Multiway Sparse Distance Weighted Discrimination

Modern data often take the form of a multiway array. However, most classification methods are designed for vectors, that is, one-way arrays. Distance weighted discrimination (DWD) is a popular high-dimensional classification method that has been extended to the multiway context, with dramatic improvements in performance when data have multiway structure. However, the previous implementation of multiway DWD was restricted to classification of matrices, and did not account for sparsity. In this article, we develop a general framework for multiway classification which is applicable to any number of dimensions and any degree of sparsity. We conducted extensive simulation studies, showing that our model is robust to the degree of sparsity and improves classification accuracy when the data have multiway structure. For our motivating application, magnetic resonance spectroscopy (MRS) was used to measure the abundance of several metabolites across multiple neurological regions and across multiple time points in a mouse model of Friedreich’s ataxia, yielding a four-way data array. Our method reveals a robust and interpretable multi-region metabolomic signal that discriminates the groups of interest. We also successfully apply our method to gene expression time course data for multiple sclerosis treatment. An R implementation is available in the package MultiwayClassification at http://github.com/lockEF/MultiwayClassification. Supplementary materials for this article are available online.

Bayesian Distance Weighted Discrimination

Distance weighted discrimination (DWD) is a linear discrimination method that is particularly well-suited for classification tasks with high-dimensional data. The DWD coefficients minimize an intuitive objective function, which can solved efficiently using state-of-the-art optimization techniques. However, DWD has not yet been cast into a model-based framework for statistical inference. In this article we show that DWD identifies the mode of a proper Bayesian posterior distribution, that results from a particular link function for the class probabilities and a shrinkage-inducing proper prior distribution on the coefficients. We describe a relatively efficient Markov chain Monte Carlo (MCMC) algorithm to simulate from the true posterior under this Bayesian framework. We show that the posterior is asymptotically normal and derive the mean and covariance matrix of its limiting distribution. Through several simulation studies and an application to breast cancer genomics we demonstrate how the Bayesian approach to DWD can be used to (a) compute well-calibrated posterior class probabilities, (b) assess uncertainty in the DWD coefficients and resulting sample scores, (c) improve power via semisupervised analysis when not all class labels are available, and (d) automatically determine a penalty tuning parameter within the model-based framework. R code to perform Bayesian DWD is available at https://github.com/lockEF/BayesianDWD. Supplementary materials for this article are available online.

Machine learning models for classification and identification of significant attributes to detect type 2 diabetes

Type 2 Diabetes (T2D) is a chronic disease characterized by abnormally high blood glucose levels due to insulin resistance and reduced pancreatic insulin production. The challenge of this work is to identify T2D-associated features that can distinguish T2D sub-types for prognosis and treatment purposes. We thus employed machine learning (ML) techniques to categorize T2D patients using data from the Pima Indian Diabetes Dataset from the Kaggle ML repository. After data preprocessing, several feature selection techniques were used to extract feature subsets, and a range of classification techniques were used to analyze these. We then compared the derived classification results to identify the best classifiers by considering accuracy, kappa statistics, area under the receiver operating characteristic (AUROC), sensitivity, specificity, and logarithmic loss (logloss). To evaluate the performance of different classifiers, we investigated their outcomes using the summary statistics with a resampling distribution. Therefore, Generalized Boosted Regression modeling showed the highest accuracy (90.91%), followed by kappa statistics (78.77%) and specificity (85.19%). In addition, Sparse Distance Weighted Discrimination, Generalized Additive Model using LOESS and Boosted Generalized Additive Models also gave the maximum sensitivity (100%), highest AUROC (95.26%) and lowest logarithmic loss (30.98%) respectively. Notably, the Generalized Additive Model using LOESS was the top-ranked algorithm according to non-parametric Friedman testing. Of the features identified by these machine learning models, glucose levels, body mass index, diabetes pedigree function, and age were consistently identified as the best and most frequently accurate outcome predictors. These results indicate the utility of ML methods in constructing improved prediction models for T2D and successfully identified outcome predictors for this Pima Indian population.

Generalized self-concordant analysis of Frank–Wolfe algorithms

Projection-free optimization via different variants of the Frank–Wolfe method has become one of the cornerstones of large scale optimization for machine learning and computational statistics. Numerous applications within these fields involve the minimization of functions with self-concordance like properties. Such generalized self-concordant functions do not necessarily feature a Lipschitz continuous gradient, nor are they strongly convex, making them a challenging class of functions for first-order methods. Indeed, in a number of applications, such as inverse covariance estimation or distance-weighted discrimination problems in binary classification, the loss is given by a generalized self-concordant function having potentially unbounded curvature. For such problems projection-free minimization methods have no theoretical convergence guarantee. This paper closes this apparent gap in the literature by developing provably convergent Frank–Wolfe algorithms with standard mathcal {O}(1/k) convergence rate guarantees. Based on these new insights, we show how these sublinearly convergent methods can be accelerated to yield linearly convergent projection-free methods, by either relying on the availability of a local liner minimization oracle, or a suitable modification of the away-step Frank–Wolfe method.

Skeletal model-based analysis of the tricuspid valve in hypoplastic left heart syndrome.

Hypoplastic left heart syndrome (HLHS) is a congenital heart disease characterized by incomplete development of the left heart. Children with HLHS undergo a series of operations which result in the tricuspid valve (TV) becoming the only functional atrioventricular valve. Many HLHS patients develop tricuspid regurgitation and right ventricle enlargement which is associated with heart failure and death without surgical intervention on the valve. Understanding the connections between the geometry of the TV and its function remains extremely challenging and hinders TV repair planning. Traditional analysis methods rely on simple anatomical measures which do not capture information about valve geometry in detail. Recently, surface-based shape representations such as SPHARM-PDM have been shown to be useful for tasks such as discriminating between valves with normal or poor function. In this work we propose to use skeletal representations (s-reps), a more feature-rich geometric representation, for modeling the leaflets of the tricuspid valve. We propose an extension to previous s-rep fitting approaches to incorporate application-specific anatomical landmarks and population information to improve correspondence. We use several traditional statistical shape analysis techniques to evaluate the efficiency of this representation: using principal component analysis (PCA) we observe that it takes fewer modes of variation compared to boundary-based approaches to represent 90% of the population variation, while distance-weighted discrimination (DWD) shows that s-reps provide for more significant classification between valves with less regurgitation and those with more. These results show the power of using s-reps for modeling the relationship between structure and function of the tricuspid valve.

Asymptotic properties of distance-weighted discrimination and its bias correction for high-dimension, low-sample-size data

While distance-weighted discrimination (DWD) was proposed to improve the support vector machine in high-dimensional settings, it is known that the DWD is quite sensitive to the imbalanced ratio of sample sizes. In this paper, we study asymptotic properties of the DWD in high-dimension, low-sample-size (HDLSS) settings. We show that the DWD includes a huge bias caused by a heterogeneity of covariance matrices as well as sample imbalance. We propose a bias-corrected DWD (BC-DWD) and show that the BC-DWD can enjoy consistency properties about misclassification rates. We also consider the weighted DWD (WDWD) and propose an optimal choice of weights in the WDWD. Finally, we discuss performances of the BC-DWD and the WDWD with the optimal weights in numerical simulations and actual data analyses.

Sparse Multicategory Generalized Distance Weighted Discrimination in Ultra-High Dimensions.

Distance weighted discrimination (DWD) is an appealing classification method that is capable of overcoming data piling problems in high-dimensional settings. Especially when various sparsity structures are assumed in these settings, variable selection in multicategory classification poses great challenges. In this paper, we propose a multicategory generalized DWD (MgDWD) method that maintains intrinsic variable group structures during selection using a sparse group lasso penalty. Theoretically, we derive minimizer uniqueness for the penalized MgDWD loss function and consistency properties for the proposed classifier. We further develop an efficient algorithm based on the proximal operator to solve the optimization problem. The performance of MgDWD is evaluated using finite sample simulations and miRNA data from an HIV study.

Sufficient dimension reduction based on distance‐weighted discrimination

AbstractIn this paper, we introduce a sufficient dimension reduction (SDR) algorithm based on distance‐weighted discrimination (DWD). Our methods is shown to be robust on the dimension p of the predictors in our problem, and it also utilizes some new computational results in the DWD literature to propose a computationally faster algorithm than previous classification‐based algorithms in the SDR literature. In addition to the theoretical results of similar methods we prove the consistency of our estimate for fixed p. Finally, we demonstrate the advantages of our algorithm using simulated and real datasets.

ESDA: An Improved Approach to Accurately Identify Human snoRNAs for Precision Cancer Therapy

Background: SnoRNAs (Small nucleolar RNAs) are small RNA molecules with approximately 60-300 nucleotides in sequence length. They have been proved to play important roles in cancer occurrence and progression. It is of great clinical importance to identify new snoRNAs as fast and accurately as possible. Objective: A novel algorithm, ESDA (Elastically Sparse Partial Least Squares Discriminant Analysis), was proposed to improve the speed and the performance of recognizing snoRNAs from other RNAs in human genomes. Methods: In ESDA algorithm, to optimize the extracted information, kernel features were selected from the variables extracted from both primary sequences and secondary structures. Then they were used by SPLSDA (sparse partial least squares discriminant analysis) algorithm as input variables for the final classification model training to distinguish snoRNA sequences from other Human RNAs. Due to the fact that no prior biological knowledge is request to optimize the classification model, ESDA is a very practical method especially for completely new sequences. Results: 89 H/ACA snoRNAs and 269 C/D snoRNAs of human were used as positive samples and 3403 non-snoRNAs as negative samples to test the identification performance of the proposed ESDA. For the H/ACA snoRNAs identification, the sensitivity and specificity were respectively as high as 99.6% and 98.8%. For C/D snoRNAs, they were respectively 96.1% and 98.3%. Furthermore, we compared ESDA with other widely used algorithms and classifiers: SnoReport, RF (Random Forest), DWD (Distance Weighted Discrimination) and SVM (Support Vector Machine). The highest improvement of accuracy obtained by ESDA was 25.1%. Conclusion: Strongly proved the superiority performance of ESDA and make it promising for identifying SnoRNAs for further development of the precision medicine for cancers.

A machine learning approach to knee osteoarthritis phenotyping: data from the FNIH Biomarkers Consortium

Knee osteoarthritis (KOA) is a heterogeneous condition representing a variety of potentially distinct phenotypes. The purpose of this study was to apply innovative machine learning approaches to KOA phenotyping in order to define progression phenotypes that are potentially more responsive to interventions. We used publicly available data from the Foundation for the National Institutes of Health (FNIH) osteoarthritis (OA) Biomarkers Consortium, where radiographic (medial joint space narrowing of ≥0.7mm), and pain progression (increase of ≥9 Western Ontario and McMaster Universities Osteoarthritis Index [WOMAC] points) were defined at 48 months, as four mutually exclusive outcome groups (none, both, pain only, radiographic only), along with an extensive set of covariates. We applied distance weighted discrimination (DWD), direction-projection-permutation (DiProPerm) testing, and clustering methods to focus on the contrast (z-scores) between those progressing by both criteria ("progressors") and those progressing by neither ("non-progressors"). Using all observations (597 individuals, 59% women, mean age 62 years and BMI 31kg/m2) and all 73 baseline variables available in the dataset, there was a clear separation among progressors and non-progressors (z=10.1). Higher z-scores were seen for the magnetic resonance imaging (MRI)-based variables than for demographic/clinical variables or biochemical markers. Baseline variables with the greatest contribution to non-progression at 48 months included WOMAC pain, lateral meniscal extrusion, and serum N-terminal pro-peptide of collagen IIA (PIIANP), while those contributing to progression included bone marrow lesions, osteophytes, medial meniscal extrusion, and urine C-terminal crosslinked telopeptide type II collagen (CTX-II). Using methods that provide a way to assess numerous variables of different types and scalings simultaneously in relation to an outcome of interest enabled a data-driven approach that identified key variables associated with a progression phenotype.

A Multicategory Kernel Distance Weighted Discrimination Method for Multiclass Classification

Distance weighted discrimination (DWD) is an interesting large margin classifier that has been shown to enjoy nice properties and empirical successes. The original DWD only handles binary classification with a linear classification boundary. Multiclass classification problems naturally appear in various fields, such as speech recognition, satellite imagery classification, and self-driving vehicles, to name a few. For such complex classification problems, it is desirable to have a flexible multicategory kernel extension of the binary DWD when the optimal decision boundary is highly nonlinear. To this end, we propose a new multicategory kernel DWD, that is, defined as a margin-vector optimization problem in a reproducing kernel Hilbert space. This formulation is shown to enjoy Fisher consistency. We develop an accelerated projected gradient descent algorithm to fit the multicategory kernel DWD. Simulations and benchmark data applications are used to demonstrate the highly competitive performance of our method, as compared with some popular state-of-the-art multiclass classifiers.

Fast Algorithms for Large-Scale Generalized Distance Weighted Discrimination

ABSTRACTHigh-dimension-low-sample size statistical analysis is important in a wide range of applications. In such situations, the highly appealing discrimination method, support vector machine, can be improved to alleviate data piling at the margin. This leads naturally to the development of distance weighted discrimination (DWD), which can be modeled as a second-order cone programming problem and solved by interior-point methods when the scale (in sample size and feature dimension) of the data is moderate. Here, we design a scalable and robust algorithm for solving large-scale generalized DWD problems. Numerical experiments on real datasets from the UCI repository demonstrate that our algorithm is highly efficient in solving large-scale problems, and sometimes even more efficient than the highly optimized LIBLINEAR and LIBSVM for solving the corresponding SVM problems. Supplementary material for this article is available online.

A machine learning approach to knee OA phenotyping

Purpose: Knee osteoarthritis (KOA) is a heterogeneous condition characterized by changes in a variety of joint tissues and driven by a number of different potential mechanisms. To date, most clinical trials of disease modifying drugs have treated all KOA as the same disease, based primarily on radiographic features, and have failed to improve outcomes. The purpose of this study was to explore machine learning approaches to phenotyping in KOA in order to better define the progressor phenotype that may be more responsive to a disease modifying intervention. Methods: We focused on identifying differences between those knees that progressed by both radiographic (loss of medial joint space width) and pain (using the WOMAC pain scale) criteria (“progressors”) and those that did not progress by either of these (“non-progressors”), as defined by the publicly available FNIH OA Biomarkers Consortium dataset of 600 individuals with 76 demographic, imaging (quantitative and semi-quantitative MRI), and biochemical variables. First, problematic variables and observations were removed, leaving 597 observations and 73 variables. The data were transformed to reduce skewness and standardized to address differences in scale. Direction-projection-permutation (DiProPerm) hypothesis testing was used to test equality of distributions (by z-score) in the Distance-Weighted Discrimination (DWD) direction as quantified by the mean difference, using 100 permutations. Loadings on the DWD direction demonstrate the relative contribution of each variable to the class differences (e.g., progressors compared with non-progressors). In addition, we explored k-means clustering to partition the observations into 2 subgroups (additional subgroups were not supported by various statistical indices). Given multiple comparisons, a z-score of at least 3 (P value∼0.001) was considered statistically significant. Results: When considering all observations and all variables simultaneously, the progressors and non-progressors clearly separated with a z-score of 10.10 (Table 1).Table 1Z-scores derived from the DiProPerm test for the difference between progressors and non-progressors.All variablesDemographicQuantitative MRISemi-Quant MRIBiochemicalDiProPerm z-scoresAll observations10.101.4711.6210.282.432 clusters16.191.496.515.722.0124.771.394.735.780.97z-scores >3 are statistically significant at P<0.001, in bold. Open table in a new tab z-scores >3 are statistically significant at P<0.001, in bold. No statistical improvement was seen with partitioning the observations into 2 clusters, although some of these clusters did have significant z-scores. We were also able to study the 4 groups of variable data separately, finding higher z-scores for the MRI variables (11.62 for quantitative and 10.28 for semi-quantitative), with lower scores for demographic and biochemical markers. Again, lesser z-scores were seen for the 2 clusters compared with using all observations. Finally, we were interested in the relative contributions of each variable to the overall difference between progressors and non-progressors (Fig. 1), where the DWD loadings were represented in a bar plot for the 40 variables with the greatest contribution. The greatest positive contribution was seen for WOMAC pain (which is part of the progression construct), lateral meniscal extrusion, and serum PIIANP; the largest negative contributions were from the number of subregions with bone marrow lesions, the number of locations with any osteophyte, and urine CTX-II. Conclusions: These innovative methods provide a way to assess numerous variables of different types and scalings simultaneously in relation to KOA progression as shown here, or for assessing any other outcome of interest. Such methodology could identify both known and novel KOA phenotypes, potentially improving patient selection for specific interventions and providing insight into pathophysiology in this heterogeneous condition.