A hierarchical multivariate denoising diffusion model

Abstract

Multivariable time-series prediction based on the denoising diffusion probabilistic model (DDPM) highlights a major challenge in improving prediction robustness and ensuring relatively low computing costs. This paper proposes a novel hierarchical multivariate denoising diffusion model called HMD-Diffusion to address this challenge. First, the model uses a robust Gauss–Newton (RGN) algorithm-based sampling strategy to reduce the sampling computation cost of the diffusion model and a Gated Attention Unit (GAU)-based mechanism to improve the multivariate prediction ability with minimal quality loss. Second, for data pre-processing, we introduce a dynamic variable coefficient embedding layer to achieve good adaptability to higher-order tensors and higher prediction accuracy in the initial stages. Moreover, the interval prediction architecture is embedded in the DDPM for the first time, which reduces the influence of interference factors in multivariate time-series prediction on the prediction accuracy of hierarchical models. The experimental results in two real time-series prediction scenarios show an improvement in the predictive performance of the proposed model compared to other benchmark models.