GMDM: A generalized multi-dimensional distribution overlap metric for data and model quality evaluation

Abstract

Listen

Translate article

In this article, we design and analyze a generalized multi-dimensional distribution overlap metric (GMDM) as a generic tool for quantifying similarity or difference between two multivariate distributions for the evaluation of data and model quality. Our experiments on different real-world datasets substantiate that the proposed metric is an apposite intuitive alternative for quantification of the performance of training models and the consistency of multiple datasets. The proposed metric is suitable, for example, for image quality assessment, evaluation of unsupervised training models trained using unpaired data, image denoising models, feature embedding in protein sequence classification, and the estimation of class-specific consistency in datasets of variable dimensions such as the MNIST, ORL, and CIFAR. Furthermore, it is substantiated that the proposed metric can serve as a batch-wise consistency score to evaluate the reproducibility of the surface-enhanced Raman spectroscopy (SERS) based molecule identification model and is also applied to the domain adaptation problem in the anomaly detection task. Our results on diverse applications confirm the generalization of the proposed method and indicate a good agreement of the GMDM with conventional data quality and performance metrics.