Leveraging VQ-VAE tokenization for autoregressive modeling of medical time series

Abstract

In this work, we present CodeAR, a medical time series generative model for electronic health record (EHR) synthesis. CodeAR employs autoregressive modeling on discrete tokens obtained using a vector quantized-variational autoencoder (VQ-VAE), which addresses key challenges of accurate distribution modeling and patient privacy preservation in the medical domain. The proposed model is trained with next-token prediction instead of a regression problem for more accurate distribution modeling, where the autoregressive property of CodeAR is useful to capture the inherent causality in time series data. In addition, the compressive property of the VQ-VAE prevents CodeAR from memorizing the original training data, which ensures patient privacy. Experimental results demonstrate that CodeAR outperforms the baseline autoregressive-based and GAN-based models in terms of maximum mean discrepancy (MMD) and Train on Synthetic, Test on Real tests. Our results highlight the effectiveness of autoregressive modeling on discrete tokens, the utility of CodeAR in causal modeling, and its robustness against data memorization.