Large amplitude vortex-induced vibration (VIV) of long suspenders, caused by vortex-shedding and wake flow, will decrease the service life of high-strength wires and its anchorage systems, hence listed as an essential monitoring indicator of the structural health monitoring system in the suspension bridges. Traditionally, a static vibration amplitude limitation is usually predefined and used to identify VIV from continuously monitored acceleration data of bridge suspenders. Due to the complexity of the starting vibration of the suspenders, it may not be able to flexibly adapt to the suspenders VIV under different conditions. In addition, transient but significant vibration events may not be captured since it relies only on static vibration amplitude limits without considering the temporal information of vibration. To address the above-mentioned issues, a novel framework to autonomously identify the suspenders VIV is proposed using the multimodal fusion techniques with deep neural networks. The main contributions of the methodology are: i) by calculating the wind field and vibration characteristic, two vibration response parameters are identified as indexes for the identification of suspenders VIV based on the significant difference method of big data analysis; ii) Two different modal information of time history images and power spectral density (PSD) sequences are used as inputs to the convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) to extract the suspenders vibration response features from the time and frequency domain perspectives, respectively; and iii) The multimodal information is concentrated and enhanced by the multi-head attention (MHA) mechanism to achieve automated identification of the suspenders VIV. The proposed framework is validated with data collected from a full-scale long-span suspension bridge in comparison with the base model. The results show that the proposed framework can efficiently identify the full-cycle development process of the suspenders VIV. This study provides a convenient strategy for suspenders VIV identification, which can be used for bridge management and vibration mitigation device design.