Driven by the requirements for a comprehensive understanding of composite dynamic systems in industrial processes, this paper investigates a new soft sensor for quality prediction based on slow and fast time-varying latent variables extraction using layer-wise residuals. First, the slow feature partial least squares were expanded into long-term dependency by introducing explicit expressions of the potential state of the process into the objective function. Then, the multilayer regression model for exploring composite dynamics driven by layer-wise residuals is developed using a serial structure that can extract both slow and fast time-varying latent variables that are completely orthogonal. Finally, the exponential-weighted partial least squares are proposed for extracting fast time-varying dynamic latent variables by learning the exponential decay properties of the time-series data correlation. Case studies on the industrial debutanizer and sulfur recovery unit show that the prediction accuracy of the proposed approach outperforms traditional methods.