In alarm systems, conventional univariate alarm methods often result in frequent false and missing alarms, calling for an urgent need to introduce multivariate information. For the multivariate alarm design, although each variable’s detection sensitivity is improved with the assistance of other variables, it is also susceptible to anomalies of other variables. Therefore, it is a challenge to properly introduce multivariate information, especially for complex processes subject to operating condition changes. This paper proposes a multivariate alarm framework that complements temporal and multimodal process characteristics to better alarm for variables by unbiased estimation. The temporal and multimodal characteristics are explored by prediction-oriented network structure and reconstruction-oriented network structure, respectively. To make them properly integrated, pattern labels that reveal the modes change are designed and used as the bridge between the temporal and the multimodal parts. On the one hand, the consideration of two types of characteristics allows perceiving temporal-related and modal-related faults, promoting sensitive alarm performance. On the other hand, unifying the two types of networks can eliminate the estimation bias of normal variables, making them unsusceptible to anomalies of other variables and promoting accurate alarm performance. Experiments on the coal mill prove the effectiveness of the proposed method regarding false alarm rate and missing alarm rate.