The physical condition and durability of cement-based structures can easily decrease after years of operation due to exposure to severe scenarios associated with internal defects, aggressive usage, and continuous load changes. Therefore, accurately assessing such infrastructures is essential for ensuring safety and serviceability and preventing hazards. In this study, an experimental investigation of the multimodal failure characteristics of cement mortar with various sand contents at different uniaxial compressive loading rates was conducted. Specifically, the multimodal failure characteristics include three main categories: mechanical properties (e.g., compressive strength), one-dimensional signals (e.g., micro-vibration and electromagnetic radiation), and two-dimensional videos/images (e.g., crack propagation trajectory). A geophysical acquisition and a high frame rate camera were employed to simultaneously record the micro-vibration, electromagnetic radiation and crack propagation videos of the cement specimens in a uniaxial compression test. Based on the computer vision technique and the feature pyramid network (FPN), an automatic pixel-level crack segmentation method was proposed to extract cracks of each critical moment in the fracture video. Then, the fractal dimension and crack area were calculated to quantitatively depict the crack propagation trajectory. Finally, the stress drop, micro-vibration, electromagnetic radiation, and crack propagation trajectory characteristics were analysed and discussed. The experimental results indicate the following. (1) The FPN offers a practical way to identify cracks in structures of cement-based materials at the pixel-level. (2) The micro-vibration, electromagnetic radiation, and crack propagation trajectory exhibit good consistency. In particular, the area and fractal dimension of cracks positively correlate with the stress drop of cement under compression loading. (3) The peak and cumulative energy values of micro-vibration and electromagnetic radiation tend to increase as the sand content increases when subjected to the same loading rate.