Burning Sage: Reversing the Curse of Dimensionality in the Visualization of High-Dimensional Data

Abstract

In high-dimensional data analysis, the curse of dimensionality reasons that points tend to be far away from the center of the distribution and on the edge of high-dimensional space. Contrary to this, is that projected data tends to clump at the center. This gives a sense that any structure near the center of the projection is obscured, whether this is true or not. A geometric transformation to reverse the curse, is defined in this article, which uses radial transformations on the projected data. It is integrated seamlessly into the grand tour algorithm, and we have called it a burning sage tour, to indicate that it reverses the curse. The work is implemented into the tourr package in R. Several case studies are included that show how the sage visualizations enhance exploratory clustering and classification problems. Supplementary files for this article are available online.