PCA-Enhanced Autoencoders for Nonlinear Dimensionality Reduction in Low Data Regimes

Abstract

Many scientific domains, such as nanophotonic design, gene expression, and materials design, are limited by high costs of acquiring data. This data is often intrinsically low-dimensional, nonlinear, and benefits from dimensionality reduction. Autoencoders (AE) provide nonlinear dimensionality reduction but are typically ineffective for low data regimes. Principal Component Analysis (PCA) is data-efficient but limited to linear dimensionality reduction. We propose a technique that harnesses the benefits of both methods by using PCA to initialize an AE. The proposed approach outperforms both PCA and standard AEs in low-data regimes and is comparable to the best of either of the two in other scenarios.