Online kernel sliced inverse regression

Abstract

Online dimension reduction techniques are widely utilized for handling high-dimensional streaming data. Extensive research has been conducted on various methods, including Online Principal Component Analysis, Online Sliced Inverse Regression (OSIR), and Online Kernel Principal Component Analysis (OKPCA). However, it is important to note that the exploration of online supervised nonlinear dimension reduction techniques is still limited. This article presents a novel approach called Online Kernel Sliced Inverse Regression (OKSIR), which specifically tackles the challenge of dealing with the increasing dimension of the kernel matrix as the sample size grows. The proposed method incorporates two key components: the approximate linear dependence condition and dictionary variable sets. These components enable a reduced-order approach for online variable updates, improving the efficiency of the process. To solve the OKSIR problem, we formulate it as an online generalized eigen-decomposition problem and employ stochastic optimization techniques to update the dimension reduction directions. Theoretical properties of this online learner are established, providing a solid foundation for its application. Through extensive simulations and real data analysis, we demonstrate that the proposed OKSIR method achieves performance comparable to that of batch processing kernel sliced inverse regression. This research significantly contributes to the advancement of online dimension reduction techniques, enhancing their effectiveness in practical applications.