T-SNE with High Order Truncation Fractional Gradient Descent Method

Abstract

T-distributed stochastic neighbor embedding (T-SNE) algorithm with high order truncation is a kind of dimensionality reduction method, which belongs to nonlinear dimensionality reduction and is a popular learning method. In the T-SNE algorithm with high-order truncation, the T-distribution is used to replace the Gaussian distribution when calculating the probability between sample points in the low-dimensional space, and the objective function is solved by Kullback-Leibler divergence and high-order truncation. T-SNE is developed from SNE, and its long tail effect effectively solves the crowding problem of SNE. However, there is no unique optimal solution for T-SNE, and there is no estimated part. The number of iterations and the convergence of the objective function are the reasons that affect the visualization effect of T-SNE. In terms of the convergence of the objective function, it is found that the extreme value of fractional order is not equal to the true extreme value of the objective function, which will make the fractional step method impractical. In order to enhance the global convergence, the new fractional gradient descent method of high-order truncation is used to iterate to obtain the optimal solution of the objective function and optimize the visualization effect of T-SNE.