HCt-SNE: Hierarchical Constraints with t-SNE

Abstract

Dimensionality reduction (DR) methods are useful when analyzing high dimensional data, in particular, if one wants to visualize them. t-distributed stochastic neighbor embedding (t-SNE), one of the most widely used DR methods, can preserve neighborhood information and reveal groups in embeddings. However, it may not preserve the global structure and fail to reveal the semantic information in the visualization. From a user point-of-view, a DR visualization is useful if it not only reveals hidden structures in the data but also corresponds to the user knowledge. This paper addresses these problems by proposing Hierarchical Constraint t-SNE (HC $t$ -SNE), a method that allows users to integrate hierarchical constraints directly into $t$ -SNE embeddings. The user constraints are encoded in an explicit tree. We transform the hierarchical information in this tree into a novel regularization term based on triplet constraints among the nodes at different levels in the tree. Our method takes advantage of semantic information provided in class labels and outperforms the original $t$ -SNE and two other supervised DR methods in terms of both visual assessment and quality metrics on three classic image datasets: MNIST, Fashion-MNIST and CIFAR10.