Asphalt concrete waterproofing layer (ACWL) based on custom-designed railway asphalt concretes has been proposed to stabilize subgrade moisture content of high-speed railways. However, given the temperature-dependent viscoelasticity of railway asphalt concretes, the low-temperature environment poses a great challenge for ACWL. Therefore, this study was motivated to perform experimental and numerical investigations on the low-temperature fracture behavior of railway asphalt concretes, to enable the material optimization of ACWL in cold regions. First, semi-circular bending (SCB) tests were performed at a low temperature of −10 °C, based on asphalt concrete prepared from two types of asphalt binders (A and B) and two aggregate gradations (AC-10 and AC-13). Subsequently, both the homogeneous 3D and the heterogeneous 2D model were established based on the extended finite element method (XFEM) to analyze the fracture behavior and the mesoscale mechanisms, respectively. In particular, an improved polygonal random aggregate generation and packing algorithm was proposed for the 2D model based on the actual morphology, which achieved reasonable accuracy and efficiency in fracture simulation. The results showed that the fracture process of railway asphalt concrete under SCB could be divided into four stages according to crack trajectory and fracture energy. In the first two stages, the crack length remained approximately 0 and the input energy accumulated in the elastic deformation. In the last two stages, the crack propagated through the specimen, and a rapid fracture energy dissipation was observed. Besides, the cracks tended to propagate upwards into the aggregate-mortar interface zone and the mastic zone for AC-13 and AC-10, respectively. Finally, the B13 exhibited superiority over its counterparts, attributed to the low-temperature deformation adaptivity of binder B and the crack deflection effect of AC-13 with coarser aggregates.